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Foreword 



rhc Multi-State ronsortiUm if deeply indebted to 
I lie authors who helped shape this publication* Their 
willingness to share their experiences and opinions 
both with the consortium and with the readers of this 
volume is greatly appreciated. The consortium believes 
that the ideas contained herein, will not only interest 
readers but also influence policy decisions* . 

The consortium is particularly indebted to the 
United States Office of Education for its support. Cre- 
ated by a Title V^rant, it has benefited both from ad- 
ditional fiscal resources and the direct involvement of 
representatives of the United States Office of Education* 
In particutar. Teacher Corps (James Steffensen), Na- 
tional Center for the Improvement of Educational Sys- 
tems (Allen Schmiedcr), and Title V (Stuart Dean) 
Ua'iK. ILwvu wiusi hupporiive • r»f ^ts eaous anu neiptui 
in planning the meeting that led to the conference, the 
source for most of the .papers appearing in this publi- 
cation* 

The Conference Committee included James Steffen- 
sen and Paul Collins (bi^tn of Teacher Corps), Bruce 
Joyce of Teachers College and the director of the con- 

iUu ^. The success of the conference and, it is 
hoped, of this publication, is in large measure a reflec- 
tion of their efforts* 
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Introduciion 



Tbc Mulii Stale Consortium is pleased to present to 
interested reattcnivthe following papers. 

We have titled this volume Assessment" because 
of the importance we attach to the assessment prob- 
lems. However, not all of the papers focus solely on 
assessment. In fact, ir is ditTieult, probably i^>ssible, 
to diiJiuss assessment without quickly touching on such 
areas as research, costs, teacher evaluation, and pro- 
gram development issues. 

Many of the articles .were presented by the authors 
to the consi>rtium at a series of meetings held in con- 
junction with the 1973 American Educational Re- 
search Association's annual meeting in New Orleans in 
February. Several of these have beeji significantly re- 
vi.sed, and we have included other chapters that 
appeared 'appropriate in the context of this publication. 

The authors' audience is primarily state education 
agency personnel and, in a larger sense, political deci- 
sion makers. However, the logic and appeal of the 
selections is not limited to that group: anyone inter- 
ested in the problems and potential of performance ed- 
ucation should be engaged by these learned and per- 
sonal opinions.. 

The first selection, "The Role of the "^State in 
Performance-Based Teacher Education-Certification** 
by Robert Roth, creates a context for viewing how var- 
ious state education agencies are approaching perform- 
ance education. Peter Airasian then explores the value 
questions that are at the heart of evaluation issues. 



Fn*d McDonald looks at the **State of the Art in Per- 
formance Assessment/' and Barak Rosenshine makes a 
series of recommendations concerning the research di- 
lemmas. James Popham in. three separate papers, 
touches on the problems of selecting assessment sys- 
tems, developing performance tests, and identifying 
minimal competencies. Del Schalock th'cn details what 
occurs when ^'Moving From Conceptualization to 
Practice in Assessment.*' 

The a.ssessmfcnt emphasis now shifts to related top- 
ics: costs and teacher concerns. Little definitive infor- 
mation exists on the costs of developing and imple- 
menting PBTE programs and assessment systems. Two 
educators (Bruce Joyce and Herbert Hite) who have 
prepared cost analyses for their respective states (New 
York and Washington) present their conclusions. Bea- 
trice Ward also discusses the cost factors involved in 
the developmental work at the Fa/West Regional Lab- 
oratory. Their collective conclusions are that programs 
will cost more, but that using what has already been 
produced will keep the costs within manageable liuiits. 

Finally wc offer two papers prepared by teacher rep- 
resentatives, Sandra Feldman and Bernard McKenna. 
Each notes theirjnterest in the potential of l^BTE 
while also revealing their most serious concerns. Mc- 
Kenna^s paper concluded the AERA symposium and 
most fittingly concludes this volume as he recapitulates 
what has come before and reactis to what he has heard. 



The Role of the State in 
Perfoimance-Based Teacher Education-Certification 



By 

ROBERT A. ROTH 



Certification and Education 

Ihc movcm.nt toward development of perform- 
ance-based teacher ejiucation and certification programs 
has cxperierccd rapid growth within the past few 
>cars. C'urrvntly there are approximately 30 states that 
arc actively involved In the study of either perform- 
ance-based teacher education or ceitification.* The 
purpdke of this pa{vr Is not to discuss the pros and 
cons of a pcTformancc-bascd program, but to describe 
and examine the issues facing a state that has elected 
to move in this direction. Germane to the: discussion 
are the changing roles and relationships that must be 
considered in order to plan effectively fo» implementa- 
tion. Any scheme tor UevciO|^»mcnt that has not consid- 
ered and accounted for the essential undcclying issues 
rkk^i being nugatory in nature.' 

Initially it may be of value to recognize the tradi- 
tional distinction between the pnicess of teacher educa- 
tion and that ot certification. Basically, teacher, "educa- 
tion serves a preparatory function, whereas certification 
selects those who are eligible for empioymcnt and 
provides them with a license. Certification traditionally 
has been a screening device, and it has been assumed 
that the state is Ihe best agency to carry out this func- 
tion. Discussion here will focus primarily on teacher 
certification as it has been a state respi>nsibiHty. 

It is interesting to note that there has been a great 
deal more resistance to performance-based certification 
than to performance-based teacher education. At a re- 
cent conference of the Regional Interstate Project held 
in Denver, the ci>nscnsus appeared to support this dis- 
tinction. Sandra I-eldman, vice president of New York 
State Local AF1\ stated **We do not oppose Perform- 

i Robert A. Rolh. "Performance-Based Teacher Certifica- 
tion: A Survey of the Slates," I renlon: New Jersey Slate De- 
partment of Kitucalion. Division of Field Services* December 
1972. 



ance-Based Teacher Education. The concept is a wel- 
come one . . . We oppose, however, a changeover to 
PerU>9^anve-Bnsed Certification at this time.'' - David 
Darland, with NBA, added, 

^ Mo^t would agree to the importance of 
the performance^ dimension of educating 

• teachers, but to establish one. prototype of 
teacher education as the sole route to legal 
licensure is pure folly^ especiallv in the ab- 
sence of established evidence through via- 
ble research. To base, advanced creden- 
tialling or renewing certifications on such 
a singular notion is evef more upsetting* 
This is not to decry experimentation with 
performanco-bastfd teacher education. Al- 
ready some developmental approaches to 
performance-based teacher education ap- 
pear promising, if not highly successful.^ 

The difference between certification and teacher edu- 
cation, however, vpries significantly depending upon 
the particular certification model. Since there are many 
ways in svhich performance-based certification can be 
structured, criticisms shbuld be centered around how 
the issues pertain to a given structure or definition of 
performance-based certification. 

If one views certification (particularly the perform- 
ance type) to be a testing procedure, then the distinc- 
tion is clear and the meaning of the skepticism is more 

" Sandra Feldman. "Pcrformance^Based Certification: A 
Teacher Unionist's View/* Per*'oimance-Based Education and 
Certification, report of the Regional Interstate Project Pro- 
gram. Denver. Colorado. July 18-20. 1972. Denver: Colorado 
State Department of Hducation, January 1973, p. 66. 

» David Darland. 'The Role of Professional Organizations 
in Performance-Based Teaclicr Kducation/* Performance-Based 
Hducation and Certiiicatior. report of the Regional Interstate 
Project Program. IX'nver, Colorado. July 18-20, 1972, Denver: 
Colorado State Department of Education. January 1973, pp. 
69, 70. 
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apparent. In ihh paradiuni* the preparing ii^stitiilions 
\a!V responsible for developing competencies in their 
teacher candidates and the state certifies a candidate's 
competence by testing him before issuing a license. 
Many difficulties associated wiMi such a licensing pro- 
cedure have been pointed out. One can argue /for ex- 
ampl»\ that there exists no empirical base on which to 
construct a valid testing technique, particularly in view 
of varied tcachin^kH^tttcxts. The problem is not the 
same with* performance-based teacher t*ducation be- 
cause there is a diversity of programs and flexibility to 
constantly develop and change the performance stan- 
'dards. Performance-based certification, it is argued, 
mandates i>nly one way of teaching, seems more of a 
finality, and is less responsive to change. 

In the approved program approach the distinction 
between certification and teacher education becomes 
less clear. When utiIi^ing an approved program ap- 
proach to cVrtification in conjunction with perform- 
ance-based criteria, as i5 frequently the case, the dis- 
tinction may become even more nebulous. Siipportrng 
performance-based teacher education but opposing per- 
formance certification then becomes a tenuous position. 
Interestingly enough, the approved program approach 
is the predominant system in use/ In 197 1 it was rc- 
p*>rted that the present time. 36 states r«?port exten- 
sive use of the approved program approach to certifi- 
cation, and in fact it has become the vehicle whereby 
forward-kn)king states have found the freedom to move 
in many promising new directions." ' 

It seems tha» the approved program approach to 
performance-based certification would be less suscepti- 
ble to criticism than tho state examination approach. In 
addition, it is the more common certification system 
currently in use and it provides a certain degree of 
freedom to explore new directions such as the perform- 
ance-based model. The project. Improving State Lead- 
ership in Education, issued a report which concluded 
that 'it would appear that the effective administration 
of a state-wide porformancc-based teacher certification 
s>>>tem would depend almost entirely upon an cfTectivc 
system for program approval.'* ' In view of these fac- 
tors, the models for perfi)rmance-based certification to 



ft>Ilt)w will he within the context of the approved pro- 
gram approach. 

Issuers 

In selecting a particular model, a number of impoi- 
tant issues need ti> be considered. An^essential question 
is what the role of the state should be in the certifica- , 
tk)n process. There are at -least two opposing 
viewpoints concerning the state's function. On the one 
hand, there are those who see the state as an adminis- 
trative and regulatory bi>dy. 

The belief, is that the state must improve 
its guardianship cf the public interest by 
setting, ever higher standards and develop- 
ing more efiicient systems of management. ' 
In one sense the state knows what is 
best.'' 

This view of the state'Jj rolejn certification is the 
predominant one currently in practice. It is a central- 
ized approach with uniformity and standardization 
being the emphasis. Even an approved prograni ap- 
proach could fit into this scheme if regulations con- 
cerning program content are specified. A performance- 
based certification system structured on the above 
tenets would specify teacher performance criteria for 
ceriilication at tnc siaic (cvci. 

The opposin**, viewpotat on certification emphasized 
a decentralized system with mpre local control and, a 
broader base fof decision tffaking and social change. Iii 
this strategy, ''the state must promote change rather 
than mandate it and accept diversity .as more respon- 
sive to the state*s needs than mandated single 
standards." • , 

The competency approach could easily fit into this 
philosophy also by allowing teacher education pro- 
grams or other professional agi^ncies to develop their 
i)wn particular sets of competencies. In fact, as An- 
drews points out ^in some places the competency 
movement "has been adopted as an attempt to reform 
the educational system by changing the locus of au- 
thority and thereby the way in which decisions are 
made." ' One result of this is that a variety of stand- 



• Improving Slate Lcaiiership in Fducation. •*HhmBi*ig an^ 
Effecting leiprovcmcnts in the Preparation and CcrtiHe*ition of 
Fducaiors." report of a Special Study. Denver, Colorado. 
Aprin971. p. 7. 



Theodore E. Andrews, '^Its Wisdom and Its Folly,'' paper 
presented to the Workshop on Probtems of C?ompetency- Based 
leacher Hducation. leacher C orps. State University of New 
York at Albany. May 1972. p.'7. 

• Ibid. 



Mbtd., p. 15. 



" Ibid., p. 6. 



aids appear, replacing the single set of state standards. 

The implementation of "a specific viewpoint of a 
state s role results in a number of r4miiticaiu)ns inhcr- 
cnt in the particular position. 'l*hcse*consequcncei arc, 
in effect, the undcrl>ing issues which impinge upon the 
decision to^ sdcct a particular state role and therefore 
should be carefully considered. 

In the centralized .view of the state's role, a set of 
performance criteria would be established at the state 
leveh These criteria may be devw'loped by a state" 
agency or throi||^h statewide improvement, the merits 
of which will be disct^jjcd at a later point. This stand- 
ard set of statewide eriiferia'^can be utilized in an ap- 
proved program approach or can be developed into a 
state testing instrument. Since the fdti^er has been de- 
termined H) be possibly more advantageous, immediate 
discu*:sion will follow in thin context; 

T|je approved program approach itself has been 
evalu;ih:4^y sonic educators as being restrictive. Lier- 
heimer has pointed out fhat^. 

thtf colleges approved program must fol- 
low exactly the courses prescribed for 
state certification. Such' a currioular re- 
quirement does, not provide the freedom 
which colleges must have if they are also 
to be held responsible for the qualifica- 
tions of the tea'^hcrs they prepare.^ 

His remarks are made particularly pertinent* to a 
competency-based program by substituting **pcrform- 
ance criteria**"* lor '^courses" in his statement. Thus, 
lack of curricular frev;dom may result from a central- 
ized state role with statewide performance criteria. 

Curricular freedom Extends beyond the right to de- 
cide on a particular set of courses. The freedom to ox- 
pcrinieht with innovative curricula also appears jto be 
precluded by. a rigid set of state performance criteria. 
The right of colleges' to experiment becomes afi^ impor- ^ 
tant issue in the selection of a performance-based certi- 
iication model. ^ 

The project. Improving Smc Leadership in Educa- 
tion, reported that critics of certification structures u\ 
general complain that *The rigidity of state ncquire- 
ments discourages flexibility and creativity in teacher 
preparation programs." Further, 'Idoaliy, the ap- 
proved program approach woul^ allow institutions to 
experiment and develop creative programs of teacher 

Alvin P. IJerhcimor. "Give Up the Ship.*' paper presented 
to the Massachusetts AtJvisory C ouncil on FdUi,\ition. Boston 
University, February !968. p. 4. . ^ 

Improving State Leadership, p. 3. 



preparation and encourage innovation in teacher edu- 
cation within the framework of generally agreed upon 
goals/* An ii]iportant part of this last statement is 
the word •'generally.!' Generally agreed upon goals 
may stilt provide the freedom that Lierheimer is con- 
cerned about. 

It would seem that the centralized view of the stateV 
role with a stam ard set of specific performance criteria 
would be' contrary to the intent of the approved pro- 
gram approach. Yet, performancC-based certification 
appears to depend "almost entirely .upon fin effective 
system for program approval.*' An approved pro- 
gram approach without highly specific criteria is an al- 
ternative. 

Curricular freedom, the right to experiment, flexibil- 
ity, innovation, and creativity in programs are issues 
related to the state's role that directly affect the teacher 
preparation Institution. Other issues relate to the indi- 
vidual and the restrictions imposed by a specific set of 
^)erfoirmance criteria exisling as state standards for cer- 
tification. 

McDi)nald relates that 'The specifics of teaching 
competence will differ markedly depending on liow we 
decide about the freedom each person will be given to 
choose the goalii and means for his personal develop-c 
inent and hiji life style.'? At one extreme thq^teach- 
cr\ services are sought requiring social skills, but at 
the other enjl he is an expert strategist requiring tech- 
nical skills. A specific set of state standards may only 
permit one of these philosophies to prevail, as options 
may be impractical or even contradictory. Yet, one 
niay argue that without state control contradictory 
standards co|;ld exist. 

McDonald also raises a related issue. "Should we 
not consider whether a teacher hasahe freedom to de- 
fihe the nature of his service to students? Does he have 
the freedom to decide what will be. required of 
him?*' Decisions on these questions* clearly have im- 
plications for standardization of competencies* and the 
role of the state. , ^ ^ 

An overriding concern' with the performance criteria 
approach is that students will be boxed-in, forced to 
conform to a particular mold. It is argued by some 
that 'XTtification must provide for flexibility in person- 

Ibid., p. 7. 

Ibid., p. 4. * . 

» * Frederick J. McDonald, *The Philosophical Problems of 
Competency 'Based Teacher Educdiion," Teacher Corps, State 
l-nivcrsily of New Yprk^af Albany, May 12, 1972, p. 5. 

'« Ibid., p. 7. 



ality, method, and philosophy, (open classrooms, tradi- 
tional, etc.). A specific set of standards at the state 
level does not provide for this flexibility. The decen- 
tralized state Vole docs, 0$ it allows diversity in pro- 
grams and pefCi^rmance criteria. 

The American Association of Colleges for Teacher 
Education in Evaluative Criteria* for Accrediting 
Teacher Education, A Source Book on Selected' Issues, 
asserts that "there are and should continue to be sev- 
eral philosophies of teacher education/' Will a cen- 
tralized state role and specified performance criteria 
prfa\ide varietl philosophies of teacher education? 
Each state must examine its particular structure to de- 
termine whether or not this would occur. 

Several other questions must be considered in rela- 
tion* to the development of a set of formance crite- 
ria at the state level. Can ^uch cuieria readily be 
changed? Can a standard set of competencies be devel- 
oped to fit all teaching situations »or must a number of 
sets of criteria be designed? In relation to th^ affective 
domain/ Elam believes 

The competencies that are easier to ^ie- 
scribe and to evaluate are likely » to domir 
nate . . . The skills of teachmg and the 
behaviors of a teacher which are difficult 
to learn and to evaluate often focus on th^ 
human aspects of teacher-pupil eoiHacts.'^** 

' Can these performance criteria be established in the 
affective domain on a state^wide basis,. or are they situa- 
tion specific and thus call for multiple standards devel- • 
oped at local levels? Will decentralization make 4he 
problem any easier to solve? 

. The arguments suggesting a need for an empirical 
base for performance-based certification but not 
teacher education were presented eai^ier. These argu- 
ments pertain to a certification system with a uniform 
set of standards at the state level, the centralized *v;ew 
of the state's role; 

At a recent meeting of the American Federation of 
Teachers, tfec following statement was issued in a re- 
port. ' 1 

. 

American Association of Coliegcs for Tc^hcr Education, 
Evaluative Criteria for Accrediting Teacher TEducation: A 
. Source Book on Selected Issues, Washington, D.C.: AACTE, 
1967, p. 26. 

^« Stanley . Elem. "Performance- Based Teacher Education: 
What is the State of the ArtT' Performance- Based Teacher 
Education Series. No. I. Washington, D.C.: AACTE, 1971, 
P 19. 



. * 

^ If state ageneies I'^gin to require the 

mastery of specific competencies as a pre- 

. . requisite for certification, two dangers 
would eyist. The ^fiijst wdwld be that 
pointefi out earlier:' Ti04» vj;datpd knowl- 
edge and skill competencies as well as per- 
' sonal charafctefistics unrelated to true 
teachmg effectiveness may be required, 
leading to certification standards perhaps 
. even more non-rllevant than those now 
existing. Second; pressure groups may be 
. able *o legislate requirements that attempt 
to define teacheVs and teacher behaviors . 
into unacceptable . patterns. A» Candidate 
could be required to ftt the mold or not be 
certified." 

Perhaps general' guidelines or a variety of standards 
developed by Ipcai- groupsr or institutions would be less 
susceptible to thesa dangers. On the other hand, these 
groups may be just as-Jikelyto comnut thesQ errors. 

In reference to establishing a minimuil^set of com- 
petencies at the state levol, Andrews surmises that 

Ev^uating thet competeacies demands.a 
frame of reference, at its' heart a set of 
Rvalues: I worry about spates establishmg / 
value systems, thus the frame of reference . . 
must be diversified and most likely local* 
ized*. . . Since we have a diverse popula- 
tion with^ varied philosophies,- 1 believp a. 
state should promote a certification system^ 
that expects diversity and challenges all to 
' meet the highest level of , accomplish- 
ment.*** 

Those who favor a uniform set of quality standards 
throughout the state, however, would seek the more 
centralized decision-making state role. Inequities 
among programs would thus be eliminated and em- 
ployers would be assured that all certified personnel 
possess at least a minimu m sc t^of competencies. 

In analyzing the models in terms of the issues,''an 
important question should always remain in sight. In 
most cases it will not be a matter of whethet or not a 
condition exists, biiV ttf ,what extent it exists. For exam- 
ple, to state that curricula; freedom does or does not 
exist is merely an ppinion that does not focus on the 
issue. The real issue is whether or not there is suf- 
ficient curricular freedom 'to satisfy those involved. 
Carrying the exaniple to the other extreme, there may 

* 

# : 

»^ American Federation of Teachers, AFT-QuEST Consor* 
tium Ycarboi*. Washington. D.C.: AFT, April 1972, p. 30. 

»» ArtSrews, "Its Wisdom/' p. 12. 
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be circumstances, that permit curricular freedom (or 
other conditions ) ti> exist to such an extent that it de- 
stroys another essential or desirable vlement of a certi- 
fication structure. The mixlels must be scrutinized to 
, determine if conditions arc sutlicient|y provided for, 
but not overindulged. 

Models 

• • • h 
TKere are.mimy ways in which a performance-based 
teacher certification system can be designed withi^i the 
approved program approach. At one end of a coatin- 
uum we have a very open system with maximum flexi- 
bility, whereas at tfic other end we have a highly struc- 
tured and ccntr;iliJied approach (figure iV. There* arc, 
'of course, many possibilities in between. Some of the 
modelri have b.*en alluded to in the discussion of Jssues. 



these standards, preparatii>p|[^rograms are to be devel- 
oped and implemented bv a consortium of agencies. ^ 
Each agency designates its^owh representative (s) and' 
darifies with that (those) representative (s) his (their) 
authority in acting in behalf* of the agcncyrThe agen- 
cies in a consortium are colleges and universities, 
•school organizations, profcssiortal associations. 
^ The professional association, deterihined by the total 
faculty of certifieJ employees in a school organization 
in accordance with state law election procedures, has 
the responsibility of providing opportunity Jot input 
from all other specialized and subject matter associa- 
tions. The school organi^tion represents parents, ibcal 
boards, and administration.* ^ * • - . • 

, The consortium is charged with describing rolgs to 
be assumfcd by the person to be grat^fed a. specific cer 
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Eig» 1. . 
Continiftira^of Models 
,for • " 
Performance-Based Ti;acher Certification 
Approved Program Approach 
... . t 



The open-ended approach may be called the process 
modeU In this* system, the state does not determine ^tHe 
content of the teacher education program. Perfonh^ce 
criteria are not established at' the state level. The pri- 
mary role of the fftate^is to define the process for <Je- 
velopmcnt of teacher education programs, stating who^ 
is to be involved and the nature of the involvement. In 
this model, the ^:tiate plays\ a more decentmlizcd voh 
with more local^'ontrol and a broader base for deci- 
sion making. 

Some states arc now operating a competency-based 
certification system consistent with this model., The 
state of Washington is a primary example was the first 
state to adopt competency-ba.sed certification, and ribw 
has an . operational program. A new set of standards 
for approval of teacher preparation progr^^nis became* 
efFective in Washington in September 1971.^^ Under 



State of Washington. "Guidelines and Standards for the 
Development and Approval of Programs of Preparation Lead- 
ing to the Certification of Schooi Professional Personnel," 
()lympi;i. Washington: Superintendent of Public Instra«ti&n, 
adopted Jufy 9. 1971. 



tificate and with identifying and statinj^the rationale 
for the Competencies lequired of persons who plan to 
perform the described rdies. The certificates will be is- 
sued^by the* state jhrough an approved consortium pro- 
gram: These standards arc themselves process and jfcr- 
forraance standards. / v 

In reference to this model it would be of little 
meaning to suppprt performance-based teacher educa- 
tion but not * performance-based certification. One 
' merely provides for the other; hence,/tliey become part 
' of the saane process. As noted eartier, the necessary 
task is to examine the various certification models in 
terms of the issues rather than compare certification ^ 
with teacher education. 

Clearly, this state has moved toward a decentralized 
structure with more local control, a broader base for 
decision making, and diversity of standarcls. Perform- 
ance standards ^rc more readily changed with feed- 
back, and probably less resistance would be encoun- 
tered in the .state. This model values optimum freedom 
for the preparing institution in relation to curtjcular 
decisions, flexibility, and creativity. In terms of the in- 
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dividual, ihcrc is the possibility, depending on the priv 
gram, -or freedom to define goals, for flexibility in per- 
sonality, method, and philosophy, Refleeting this 
viewpoint, William Drummond, a former associate in 
the Washington State Department of Education, urged 
that ''State departmtMUs of education, therefore, should 
foster creativity and intellectual freedom and promote 
programs of teacher education which support and cher- 
ish uniqueness and individualism/' 

Washingtjin niod?!, therefore, also rejects the 
regulatory role of a state department of education. 
Wendell .Allen, as Washington'?; assistaat superintend- 
ent of public instruction, concluded 

To emphasize this regulatory role is to 
protect the status quo. When the rule is 
the thing, change must come before there 
can be a new rule. There is danger in this 
circumstance that the major er5erg!es of 
th^gency wiP be spent on administrative 
1 atnef than leadership functions.^* 

An essential point to note is the prevalence of multi- 
ple standards, lack of uniformity, less legal need for an 
empirical base, and no single set of standards. Should 
all of the above factors be deemed advisable, then a 

V ' particular state might select this model. 

''^ New York has envisioned a very similar type of 

program.-' Four process standards have been estab- 
lished to be utilized for the development of pilot proj- 
ects. The standards require the establishment *of a pol- 
icy bi>ard made up of representatives' of teachers, 
school districts, colleges, and teacher education stu- 
dents. This group considers the objectives of the 
schools involved, the competencies teachers need to be 
successful in that environment, as well as those quali- 
ties desirable for all teachers, and acceptable evidence 
for attainment of competencies. The policy board then 
will establish individualized programs for the prepara- 
tion of teachers to meet these criteria. Finally, a man- 
agement system must be established. Trial projects may 
be designed for initial or continuing certification or 

2'^ William H. Drummond. '•Conference Commentary'* in 
The Seattle Conference: The Role of the State Department of 
Education in Teucher Education, Olytftpia, Washington; State 
Supcrinfcndent of Public Instruction, 1967, p. 74. 

2» Wendell C, Allen, "State Government and Teacher Edu- 
cation -A Different Role for the State Education Agency in 
The Seattje, Conference, p. 78. 

New York State Educaiion Department, "A New Style of 
CAtificaiion.'' Albany, New York: New York State Depart- 
ment of I'ducation, Division of Teacher Education, March 15, 
1971. 



both. The State Department of Education will exercise 
its legal responsibility for program approval. Note the 
decentralized role and the belief that performance cri- 
teria are mostly situation specific. 

Vermont has expanded the decision-making base 
to kKal schixil districts. A local school district may de- 
velop a program for the inservice training and profes- 
sional advancement of its staff and may apply to the 
State Department of Education for approval to recom- 
mend issuance and renewal of all certificates at the 
local level. The appropriate certificate will be issued by 
the State Department of Education. 

The local district must submit evidence that the 
teachers, school board, and administrative personnel 
have participWd in the planning and development of 
the program. The local program must include provision 
for Job description, task analysis, and performance cri- 
teria for all education personnel. An approved program 
approach is ?n effect for college teacher preparation 
programs. ^ 

Washington, New York, and Vermont are*case stwt- 
ICS that fall into the process model. Local decision 
making characterizes these attempts, assuming what is 
acceptable in one situation may be unacceptable in an- 
other. 

Moving slightly along the continuum away from the 
prdeess model but within the local decision-making 
framework, there is a model suggested by Lierheimer " 
which we may call the informational model. The central 
thesis of Lierheimer^s proposal is that the staters role 
is not to make judgments but to maintain records. He 
suggests the students be tested over a multitude of 
factors including actual teaching performance. There > 
is a possibility here for utilization of performance 
criteria, but the testing is not donf by the state. 

Decentralization is emphasized ir^^^^af^proach with 
local school teams conducting the evaluation of the 
competence of potential teachers. Ultimately, the 
agency \o decide on teacher performance for licensing 
purposes would be the school. The function of the 
state is to monitor the local evaluation but not impose 
state standards. Although evaluation systems would be 
approved by the state there would be no uniform tech- 
niques for verification of classroom performance. The 
state ofliicc would maintain a data bank on all leaching 
personnel in the state. 

• * Vermont State Department of Education, "Regulations 
Governing the Certification of Educational Personnel," Mont- 
peli^r: Vermont State Board of Education. Department of Ed- 
ucation. July 1. 1971, 

^« Lierheimer* "Give Up The Ship.'* 
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A unique feature of this model is that the state ac- 
eumulates infi>rmalion on an individual but makes no 
decision in refereace to compeienee. I he major role of 
the slate is to provide resourees. The local district is 
provided with the information, and it is at this level 
that decisions are made as to whether the individuaPs 
competence fits the particular situation. The utidcrlying 
assumption is that vahies and eompefeneies arc situa- 
ti;>n specific and hence require local evaluation. Cur- 
rently, there are no states utili/ing this informational 
modcL Again, analysis of the model should be made in 
terms of all the issues identified earlier. 

This model can be modified to interject more state 
control and greater uniformity. Minimum standards 
could be set by the state for the various competencies 
or groups of competencies. These minimum standards 
v/ould be .established for certification purposes. The 
state would still maintain its individual data bank and 
Iwal districts could use the information for hiring pur- 
poses. This modified model would be farther along the 
continuum in lerms of state control and decision mak- 
ing: 

.Another open-type model which does not provide 
quite as broad a dccision-maWng bav*^ is being devel- 
oped by the state of Florida. In this case consortia are 
not designed for purposes of initial certification al- 
though inservice ^n^grams arc developed by local dis- 
tricts. This ••facilitation modeP* utilizes the college ap- 
proved program approach commonly in practice among 
the states. 

The program approval regulations are somewhat 
• process in nature indicating prescribed activities, but 
they are CiMitent standards as well, identifying courses 
necessary for certification. There are alternatives to the 
content regulations which provide for performance- 
based programs. 

An institution may, instead, specify the 
competencies which its graduates will be 
expected to demonstrate, identify the prft- 
cedures by which those competencies will 
be measured, and then develop a program 
which leads to those competencies. Once 
such a program is approved, its graduates 
will receive regular teaching certificates 
with no penalties. Institutions are now 
being encouraged to develt^p competency- 
based programs. ■ 

In this model control is in the colleges, but direction 

- • Florida Department of FJueatien, *The Florida Program 
for Improving the Training. I'valiialion. and I.icensurw of Ftl- 
iicational Personnel.** draft number 2. Tallahassee: Idnca- 
tiotial Rcseaich and Development Section. April 7» 1971. 
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is provided' by the state. The colleges de\elop their 
own conipetencies, but these are consistent with state 
ciHirse requirements. I here is additional direction and 
slinuiUis provided by the state, however, which facili- 
tates dcvelopmcrti of such programs. The state is com- 
piling a catalog of teaching competencies which will 
eventuallN be validated through research. These compe- 
tencies, or performance criteria, will be provided to the 
colleges to facilitate their program development. These 
particular criteria, however, will not necessarily be 
mandated and certain! all will not he required of a 
given institution. Othei facilitating procedures by the 
Slate are assembling of training materials based on 
{KTformance criteria and staff development for teacher 
trainers. The emphasis is on facilitation. Decision mak- 
ing IS so'mewhat diffused but the role of the state is 
stionger than in previous models. The facilitation 
model presents difTerent responses to the issues. 

The remaining two models to discussed can be 
grouped under a heading of central decision making. 
The first two models, you may recall, were local deci- 
sion-making types» with the facilitation model being 
somewhere between. These last two niodeK.are at the 
other extreme end of the continuum* 

One approach to performance-based certification h 
to establish performance criteria at the state leveK This 
approach supports a strong state ro\c and a uniform 
set of standards. It guarantees that each certificated in- 
dividual has at least a minimum set of eompetencies. 
These criteria could be utilized as a state test or part 
of an approved program. The focus here, however, is 
on the approved program approach. 

The manner in which these criteria are slated signifi- 
cantly affects the impact they will have on teacher edu- 
cation programs and the role of the state. The per- 
formance criteria can be stated in generic terms which 
then serve as guidelines for further specification by 
teacher preparation institutions. This guidelines model 
increases centralized authority yet does provide a cer- 
tain degree of participation on the part of the colleges 
or consortia, 

.As an example of competencies consistent with the 
guidelines model, it might be required that the teacher 
candidate^dcmonstratc the ability to diagnose areas of 
student deficiency, maintain a classroom environment 
which motivates students to learn, plan an instructional 
unit, employ a variety of instructional techniques, etc. 

Utah recently adopted at the state level a set of 

Vlixh State Board of Education," Reeommended Profi- 
cieiKv iMiidelines fir Media Indotsemcnls." Salt I,ake City: 
Utah State Board of Education, 1972. 
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Robert A. Roth 

performance criteria for instructional media, some of 
which approximate the guidelines model type of crite- 
ria. Hrercqui.iite to a Basic Media Endorsement are a 
bachelor's degree and a teaching certificate, An exami- 
nation foi proficiency conductefd by a recommending 
institution (with an approved certification program) is 
then administered. The recommending institution is 
free to determine how the competency will be demon- 
strated or ascertained, but a candidate may request an 
4>ppt>rtunity to demonstrate a competency whenever he 
feels he Is ready. Competencies may be demonstrated 
one at a time. Candidates who perform satisfactorily 
will be considered as having met the endorsement re- 
quirement regardless of the route taken to obtain the 
competency. 

Proficiency must be demonstrated in five areas. 
Some examples of performance standards are as fol- 
lows: 

Using media selection tools of his choice, the can- 
didate will identify the tools he has selected and 
include a rationale for the choice of each. 

— The candidate will explain what one would do to 
select new subject headings for materials which 
are n^ considered in "Sears List of Subject Head- 
ings." 

The candidate will demonstrate proficiency in 

mounting pictures by producing one acceptable 
example of the following: 

( I ) dry mount on a hard surface, using dry 
mounting tissue 



(2 ) dry mount, using dry mounting cloth 
{ 3 ) rubber cement mount 
(4) laminate with thormo copy machine, adhe- 
sive acetate, or heat press. 

At the extreme end of the continuum we have what 
can be termed the "prescriptive model." In this system 
the state provides very specific performance criteria 
(behavioral objectives) which are utilized by the col- 
leges as objectives and evaluative criteria. This is the 
most dominant of the state roles within an approved 
program approach with an emphasis on the administra- 
tive and regulatory function of a state education 
agency. Uniformity in certification with a single set of 
standards is the essential feature. 

The state of New Jersey is currently studying the 
feasibility of such a performance-based certification 
system. Specific performance criteria are being devel- 
oped for use on a statewide basis as certification stand- 
ards. How specific these will be has not as yet been de- 
termined. There are two unique aspects to the New 
Jersey approach, however, that broaden the base of de- 
cision making. The performance criteria are being de- 
veloped by task forces composed of a crosssection of 
educators from across the state, representing teachers, 
administrators, college students, college professors, the 
State Department, and the various professional associa- 
tions. These criteria, therefore, are not developed by 
the State Department but represent a consensus of pro- 
fessional educators in the state. In addition, evaluation 
of prospective teachers may involve schools, colleges, 
and professional associations, a resemblance to the 
consortium idea. 

Clearly, the statewide involvement in development of 
criteria adds considerable power lo the approach. It 
presents a decided advantage over development of cri- 
teria by a state department or even a college or univer- 
sity. It appears to have greater validity and is more 
likely to find statewide acceptance. Significantly, "it has 
been generally agreed that whoever determines certifi- 
cation requirements controls the program of 
preparation." Thus, in this instance, control is more 
in the hands of the total profession. 

The guidelines model can be developed by the same 
method. The difference between the two models then 
lies in the specificity of the criteria. How does this dif- 
ference relate to the i-ssues, and how do these two 
models compare with the open end of the continuum 
in terms of the issues? 

" Improving State Leadership, p. 5. 
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In (he pr(k»css niodcK teacher preparation institu- 
tions have maximum curricular freedom. 'Vhc guide- 
lines mode! allows the institutions the opportunity to 
develop the speeifie fx^rformance criteria while the pre- 
scriptive model does noi pri;vide for this. A compari- 
son of performance criteria with traditional course list 
standards may be of value at this point. A course in 
tests and measurement is a familiar requirement i i the 
course list system. The guidelines model would require 
competencies that arc .somewhat more specific^ such as 
ability to evaluate student performance and ability to 
develop tests. The prescriptive model, however, would 
list a number of specific performances such as ability 
to formulate es.say (multiple choice, etc.) test items 
and analyze tests for validity. Also» the evidence ac- 
cented thai the performance had been achieved would 
be provided. Omtiiuiing our compaiisi^n, if ased in a 
c<^urse list .Kysiem, a prescriptive model would list the 
things that should be taught in a tests and measure- 
ment course rather than leaving this to the college. 

Andrews has stated that "a required set of perform- 
ance criteria could be just as moribund as rigid course 
reqinremcnts have been in the past." i: appears that 
the more specific the criteria the less freedom that ex- 
ists. Recall that the approved program approach works 
**within the framework of generally agreed upon 
goals/' - ' The possibilities for creativity through inno- 
vative programs can be achieved in the design of 
means to achieve the objectives, but not through alter- 
native objectives. Two basic questions are at hand. 
Firsts is curricular freedom seen as being of value; and 
second » does a prescribed set of sprciiic performance 
criteria significantly limit this freedom? A related 
question is whether or not the guidelines model offers a 
great deal more freedom than the prescriptive model. 

A concern similar to the question of freedom is di- 
versity. I he process model allows, and even encour- 
ages, diversity among prt>granis. Those in favor of di- 
versity argue that there are varied philosophies of 
education requiring different teaching models. Any set 
of pcrtormanci criteria is ba.scd on a theory of teach- 
ing and the leaching-learning prtKcss. Although not al- 
ways articuJntcd the purposes of teaching arc inherent 
in the criteria. 

In the process model several teaching philo.sophies 
exist simultaneously with validation and development 
being ongoing procvS.ses. A .set of specific criteria, how- 

Theodore Andrews,, New Directions In Certification, 
Washin^jton. D.C.: AssiKiution of Teacher Hducators. 1971, p. 
10. 

Imp.'oving Stale I eadership^ p. 7. 



ever, relics on (Mie teaching model and also establishes 
a particular value system. The problem here is that 
there is no empirical' base to lead us to the correct 
model. As noted earlier, lack of an empirical base is a 
primary concern with perfomiancc-based certification. 
With a variety of program types, it can be argued, we 
recogni/e the developmental state of our knowledge 
base, whereas a single model .seems a finality and de- 
mands empirical validation before being adopted. This 
accounts for the suppor: of performance-based teacher 
education instead of certification. 

/Another point made by tho.se favoring diversity is 
that performance criteria are situation specific. There 
are numerous contexts for teaching, both in terms of 
environment and educational philosophy. This requires 
different .sets of competencies, at least m terms of the 
general situations (not for every school, etc.). There 
may not be enough in common to establish at least a 
minimum core of competencies at the state level. 
Washington, Vermont, and New York appear to be- 
lieve in this as evidenced by their pr<x:ess models. 

All of the above factors suggest multiple standards 
and diversity of programs. The initial question is 
whether these are valid concerns. The other position 
argues for more standardization and quality assurance. 
hy*quiti\s among programs are diminished. Certainly, 
the prescriptive model adheres to the latter viewpoifit. 
Ihe guidelines model docs provide a certain degree of 
variability in that each institution can define the spe- 
cific criteria to fit its needs. The prescriptive model in- 
sists on a single standard; the guidelines model offers 
some degree of multiple standards although minute 
when compared to the process model. 

A frequent criticism of competency-based programs 
is the problem of writing performance criteria in the 
affective domain. This problem becomes amplified as 
we move across the ciintmuum toward the prescriptive 
model. As an example, the guidelines model might re- 
quire competence in developing teacher-student rap- 
port. Each icachcr pivparation institution would be 
provided the freedom to determine not only how this 
might be developed but how it might be judged to 
exist. The prescriptive model, however, would specify 
the performance criteria necessary to achieve this, 
such as "uses student names/' or "smiles or acknowl- 
edges student responses by nodding.'' The question is 
whether or not such criteria can be written on a state- 
wide level. Ignoring the affective domain and con- 
centrating on the cognitve and psychomotor would not 
be a viable alternative. 

rhe reader may recall the issues raised concerning 
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iho righis of ihc im'ividual as suggested by Mc- 
OonalJ. ■• Atv iIkmv opporiiiniiios for flexihiliiy in per- 
si)nal t\\ ineihoil. and philosophy? What about the 
light of the individual to dciine his own glials? Raekley 
and Miller, as members of the Pennsylvania State De- 
partment of Hdueation, stated that 

Individual difTerenees arc not taken Into 
aeeouni in blanket eertiftealion standards. 
Wc are eonvineed that the irrprovemcnt of 
teaehcr preparation nuist take plaec at the 
point of initial preparatii>n . . , with atten- 
tion directed ti^ individual iieeds within the 
eonicxt of general eertitleation require- 
ments. * 

The pnKess model provides for individual flexibility, 
and, there arc functioning programs whieh operate on 
these premises. I he prescriptive model precludes much 
of thi.s, at least in terms of the specific criteria required 
by the state. I hc individual does not have the freedom 
to define his own goals, but he may have the opportu- 
nity to select his own method of achieving the objec- 
tives. Again, those favoring a uniform set of standards 
would find individual selection of goals to be undesira- 
ble and detrimental to certification. 

1 he guidelines model may provide a certain degree 
of individual choice but within the boundaries defined 
at the state level. The general objective must be ac- 
complished, but the specifics can vary with the individ- 
ual. The n)anner in which one wishes to develop teach- 
er-student rappi^rt, or plan for a lesson can vary 
significantly from another individuaPs method. The 
basic question is not just one of uniform standards ver- 
sus flexibility but the degree of each that is desirable. 

Alternatives to Approved Programs 

The discussion of issues and alternatives has thus far 
been limited to the approved program approach to 
state certification. Approved programs referred to those 
developed by colleges alone or by consortia. The evi- 
dence presented earlier in this paper suggest that ap- 
proving programs is the more viable approach to per- 
formance-based teacher certification, and some s|x:eific 
criticisms of the state testing approach were described. 

There are some teaching areas, however, that find 
themselves less rigidly tied to college preparatiuvi pro- 
grams and thus are more amenable to alteniatwe ap- 
proaches, rhe area of vocational education, for cxam- 

^" McDonaUl* ^Philosophical Problems." 

'» J. K. Raekley and Norman Miller* "Broad Policy Con- 
cerns and Direction for a Stale Department of f dilation in 
Teacher ndueution/* in Ihe Seattle Conference^ p. 15. 



pie, is somewhat unique in that it usually relies on 
expv-rietwed professionals in the various trades to enter 
the leaching pn)fcssion. 1 here are other areas, such as 
luusic, that also require specific skills unique to the 
particular profession. Educational fields such as these 
warrant consideration of alternative approaches that 
are not necessarily bound by college degree programs, 
1'hese different approaches are not neees.sarily limited 
in application to the special teaching areas mentioned, 
however, as the alternatives may be utilized for any 
teaching field if desired. 

A ct)!nmonly discussed ulternative to leachcr certifi- 
cation is the establishment of a state testing procedure. 
There arc several ways In which this can be imple- 
mented, some of which will be described here. A co- 
gent argument against this approach (which was 
pointed out earlier) is that there exists no empirical 
base on whieh to construct a valid testing technique, 
particularly in view of varied teaching contexts. The 
predictive validity of any such examination device 
would have to be established. 

It is again important to consider how the state test- 
ing models reflect the various issues. Questions about 
curricuiar freedom, jndividual freedom, and varied 
teaching philosophies should not be forgotten. The 
state testing approach to certification offers radically 
different respt>nses to the issues when compared to the 
models within the approved program approach. 

rhe informational model suggested by Lierheimer 
can easily be modified to fit a stace testing procedure. 
A set of behaviorally stated competencies could be 
formulated as certific.tijn descriptors. A teacher candi- 
date's degree of accomplishment of each of the criie/«a 
coulu be indicated to form his competence profile. 
Minimum :.»andards established for certification could 
be set by the state for each criterion or group of crite- 
ria. A system could be established (total score, 
weighted scores, etc.) to determine the individual's eli- 
gibility for certification. The state would still maintain 
its individual data bank and local districts could use 
the information for hiring purposes. 

\n important modificatioh of the Lierheimer infor- 
mational model is that not only are minimum levels es- 
tablished for certification, but the testing of the candi- 
date to determine his achievement of each criterion is 
done by the state, not wfough an approved program 
approach. The control of standards and verification of 
accomplishment reside in the hands of the state. 

The modified informational model is but one varia- 
tion of the state testing concept. Any outside agency or 
group of evaluators could be designated by the state to 
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cany oul the testing functiiui. Ihcrc is an oppi>ruinity 
to invohc nu'inbors i>f the profession in both develop- 
ineni ot eiitena and sersice on csaluaiiu^ boards im* 
teams who certify indi\idua!s. liistcad of a profile, veri- 
ticalion of minimum compcleiice miglu be all that is 
necessary. DitTerenttaied certification could be based 
on different degrees of accomplishnuMit or even differ- 
cnl types of criteria. J-valuaiing binirds or teams co«ild 
again be used throughout the entire process. 

It is generally assumed that the evaluation for certi- 
fication would be done in a live classroom situation. 
An alternative would be to establish testing centers 
where specitic skills would be evaluated such as those 
found in microteaching. This might be particularly use- 
ful for initial certification due to the inequities in stu- 
dent teachiiig situations. Students could also be used in 
test centers similar to the laboratory schools. Ihis 
would provide a more controlled situation and fewer 
variables woulil enter into the evaluation. 

\ combination of evaluation in student teaching set- 
tings and contn>lled laboratory situations is also an al- 
ternative. This might be built into a system when* a 
recommendation from a preparing institution (college 
or consortium) in addition to testing in a center would 
be necessary parts of the process for certification. The 
variations to this testing approach are too numerous to 
be included in this discussion. 



r.ach model miist be considered carefully in terms of 
the issues identified. Certainly, there are other issues to 
be acci>unted for which were not discussed here. I'he 
idea oi certification levels was not presented in this 
paper and could by itself be an entir.^ area of discus- 
sion with direct bearing on the selection of models. 
Another important question is whether or not to use 
student outcomes as an indication of teaching compet- 
ence. Concerns of a practical nature such as cost, over- 
all feasibility in terms of management, state size, diver- 
sity, and available resources are .e^i^niplos of other 
issues. The questions raised here were more of a philo- 
sophical nature and are pertinent to decision making. 

The models described were identified as being along 
a continuum. This implies that there are many other 
models which can be considered, but they most likely 
will differ from these models in degree rather than 
basic type. Perhaps ^ system can be developed with 
positive elements from several of the models described 
here. It may also be po.ssible that more thpn one model 
can be in operation at a given time, particularly if one 
accepts the notion that certain aa^as require or more 
readily fit into a state testing approach while all other . 
areas fit one of the approved program models. The 
overriding concern is which model or models best serve 
the purposes of certification. 
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Performance-Based Teacher Education: 
Evaluation Issues * 

By 

PETER AIRASIAN 



Introduction 

In this paper I shall consider two levels of evalua- 
tion in performance-based teacher education models. 
The first concerns the evaluation of the basic concept 
of performance-based teacher education. The second 
relates to evaluation issues within the context of an 
ongoing performance-based approach. I shall argue that 
the important evaluative issues are not those of select- 
ing appropriate and objective measuring techniques but 
rather involve the types of judgments which dictate 
who will be evaluated and on the basis of what per- 
formances. Those who are seeking cookbook, practical, 
**how to** answers from my presentation will be disap- 
pointed. My purpose is to argue that "how to" prob- 
lems, are predicated upon a prior set of evaluative 
decisions* decisions ultimately more important and 
powerful than any set of evidence gathering techniques, 
no matter how objective or complete these are. 

The majority of evaluation issues raised by perform- 
ance-based teacher education are not new. What is new 
and significant abc»ut performance-based teacher educa- 
tion is that for the first time, systematic evaluation of 
teacher training models and products is called for. It is 
the required synthesis of evaluation problems which 
heretofore have been treated largely individually which 
makes evaluation an issue in performance-based 
teacher education. 

Evaluation 

We must start with a definition of evaluation* since 
the term has multiple meanings according to the con- 
text within which it is considered. Perhaps the most 
commonly understood meaning of the term is '*a judg- 
ment of the extent lo which learners have mastered the 
objectives of instruction.** While this definition is ade- 

* Invited paper, Multi-State Con^rtium on Performance- 
Based Teacher Fducation« New . Orleans, [.ouisiana, February 
26, 1973. 



quate within the context of many instructional pro* 
grams, it is limited in three senses. First, it narrows 
attention to the intended outcomes of instruction. Sec- 
ond, it largely ignores information gathering about the 
process of instruction. Finally, it makes the learner the 
sole object of evaluation. To limit evaluation of per- 
formance-based programs to student evaluation is to 
overlook many of the nonlearner aims and assumptions 
inherent in the approach. It is important, particularly 
in the early stages of acceptance and implementation, 
to articulate all the evaluative issuer inherent in per- 
formance-based progr ams . To unde r take this task, 
evaluation must be recognized as encompassing a more 
general purpose than simply determining learner mas- 
tery of course outco.nes. In the context of this paper, 
then, evaluation wil! refer to ''a value judgment of 
merit or worth." Clearly this is a more general defini- 
tion than that discussed above. It is meant to be. Omit- 
ting referents such as the learner, the curriculum mate- 
rials, or the instruction from the definition focuses 
attention upon the process involved in performing an 
evaluation; that is« making a value judgment. It also 
suggests that whenever judgments are made, regardless 
of the referent* evaluation has taken place. 

1 he heart of the evaluation process, then, is valuing. 
Data gathering, be it **hard,** objective data or **soft," 
impressionistic data, is not evaluation. Evaluation takes 
place when data are ccnnpared to some standard or 
norm and a decision about pass or fail, accept or re- 
ject, or good or bad is made. For example, the fact 
lha: about 85 percent of the age cohort in the United 
States completes secondary schcH^l does not« in and of 
itself, convey value. Some of my colleagues, when pre- 
sented with this fact say ''Isn*t it good that more young 
people in America graduate from secondary school 
than in any other country in the world?'* Other col- 
leagues respond ''Only 85 percent — and 1 thought we 
were doing much better.** Or, to select an instance 



closer to performance-based teacher education, con- 
sider a pros{X'ctive teacher who teaches 60 percent of a 
group ot sixth graders the dilTcrcnec iKtwecn simple 
and eon^pinmd sentences. Is mastery by 60 percent of 
the class grounds for congratulating or chastising the 
teacher? It is only when data are judged in terms of 
standards or norms that one can ascribe value to per- 
formance. There is, in essence, a difference between 
measurement (gathering data) and evaluation (placing 
a value on the {XTformance). 

Despite our wisjies to the contrary, decisions about 
what is g(HHl, valuable, worthy, and desirable are made 
not in the research domain, but rather spring from our 
individual philosophies and frames of reference, if I 
were to ask you to cite three examples of school prac- 
tices which are based firmly in eiitablished research 
findings, 1 dare say that you would be hard put to re- 
spond. Arguments for and against performance-based 
education reside, and will continue to reside, in the 
value domain. The questions asked are not whether 
one can state performances in behavioral, measurable 
terms but whether one should state them in any terms; 
not whether teachers can be evaluated on the basis of 
their students' performance, but whether they ought to 
be. These are i.ssues which are based in philosophy and 
value orientations, and it is in these frameworks that 
evaluation ceiiters. As a consequence, it is important to 
consider evaluation issues as they relate to arguments 
for and against the basic idea of performance-based 
teacher education. 

Evaluation of the Perf ormance*Based Model 

The antecedents of performance-based teacher edu- 
cation arc many, but they all revolve around a single 
central theme : accountability. There is a serious break- 
down in the interest and ability of American citizens to 
support education. Educators are being forced to ac- 
cept responsibility for their activities and products. Ev- 
idence of success — and if not success at least efficiency 
— is becoming a prerequisite for continuing financial 
support. The "cult of efficiency and rationality" is with 
us again. "Demonstrated value for dollars expended" 
and 'it is time to make education scientific" are the 
watchwords. 

Now, my experience tells me that these are the 
watchwords of legislators and administ^^ators, not of 
classroom teachers. Accountability is, of course, always 
threatening lo those who are to be held accountable. 
There is growing evidence (e.g., Jackson, 1968; Good 
and Brophy, 1973) that classroom teachers simply 
don't think of accountability in the same rational, 
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efticiency-oriented terms as their administrative superi- 
ors. Moreover, classroom teachers perceive acetmnta- 
bility, rightly or wrongly, primarily as a method of 
pointing the finger cf blame at individu;»ls, rather than 
at a myriad of interrelated and »ditticuU-tc-dcfine fac- 
tors. ' 

Performance-based, teacher education i.s here today 
not as the result of a ground-swell of support from 
teachers, hut because administrators and politicians be- 
lieve it is a more rational, efficient, and areountablc 
method of training and certifying teachers, as well it 
may be. Performance-based programs are with us be- 
cause they are believed to be better by those who have 
some say in the matter, not because research evidence 
overwhelmingly indicates their superiority. The deci- 
sion has proceeded from the top downwards. In short, 
the values implicit in a performance-based approach lit 
closely with the values of legislators and administra- 
tors. An implicit evaluation has, therefore, already 
taken place. The evaluation was not based upon hard 
data, but rather upon a comparison of the perceived 
values inherent in performance-based versus other ap- 
proaches to teacher education. In a number of states it 
is clear that the performance-based approach has been 
evaluated as superior. 

The benefits claimed for a performance-based ap- 
proach clear and public specification of ends, emphasis 
on exit not entry behaviors, individualized instruction, 
accountability, etc. are well articulated. It is the expec- 
tation of these outcomes which affords the justification 
for current programs. Or, perhaps more appropriately, 
it is the value placed upon these outcomes which af- 
fords the justification. However, in accepting the per- 
formance-based approach, one is making a series of as- 
sumptions about teaching, teacher training, and teacher 
evaluation. It is important to identify these assumptions 
because when performance-based teacher educatbn is 
criticized and discussed, it will be primarily on the 
basis of its inherent assumptions 

Before elaborating these assumptions, it is appropri* 
ate to pause here to reemphasize the main point of my 
argument thus far. In the social sciences, arguments for 
and against innovations and practice are based primar- 
ily upon the perceived values embodied in the 
innovations and practices. Because the social sciences 
have few, if any, paradigms or models which are uni- 
\ersally accepted, social scientists wage their battles at 
the level of first causes or starting points. The physical 
sciencc.> are diffeitent from the social sciences insofar as 
the physical sciences contain laws, theories, and princi- 
ples to which all physical scientists subscribe. One phy- 
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sicist may not agree with ani;ther*s application or re- 
search in the area of relativity theory, but both accept 
the basic axioh.s t»f the theory. Such is not the case in 
the social sciences* We arc not at the stage of arguing 
about applications, but rather about star Jng points. We 
have five theories of personality, not one; three theo- 
ries of what the *'good** teacher is like, not one; four 
learning ihcorioK, not one. Each of us accepts difTercnt 
starting points about the nature of education, of man, 
or of the good. However, despite our rush to emulate 
the physical sciences, one of the most powerful aspects 
^ of the social sciences may be its pluralism. When. we 
argue and criticize it is more often about assumptions 
than about application or practice. Fo a humanistic ed- 
ucator pertormancc-based teacher education iis anath- 
ema regardless of whether it succeeds or fails because 
it emphasizes the wrong things. The point of this 
rather lengthy digression is that evaluation of porform- 
ance-based teacher education by those outside the fold 
will center upon the assumptions inherent in the per- 
formance-based approach as viewed from their value 
premises. Adherents to the movement require no evalu- 
ation of the central assumptions. They have already 
evaluated rnd endorsed them as good. Note that while 
performance-based proponents acknowledge the need 
for research about teaching activities and outcomes to 
improve the approach, few argue for a test of t!ie basic 
idea itself. The basic idea has already been evaluated 
arid accepted. 

What, then, are the assumptions inherent in the per- 
formance-based approach? Major assumptions number 
five and are as follows: 

1. There exists a set of performances which is im- 
portant for all teachers to possess. 

2. It is reasonable to identify, define, and set stand- 
ards for the relevant performances. 

3. Once identified^ we possess the knowledge and 
skill to teach the a'levant performances. 

4. Teaching can be characterized as the sum of the 
defined performances. 

5. The performances are measurably related to stu- 
dent Ipaming. 

The fact thai A have called these five propositions 
assumptions does >iot imply that proponents of per- 
formance-based teacher education consider these to be 
assumptions. In fact, the extent to which any state, lo- 
cale, or university has adopted a ^rformance-based 
approach is th«! extent to which these propositions are 
not assumptions, but givens. While thoughtful advo- 
cates of performance based teacher education recognize 
the heed for more research in areas touching on these 
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five :reas. most advocates accept the statements as 
factsT* awaiting certificatior by research. To crhics of 
the performance-based notion, hovever, the five state- 
tncnts i^re regarded as assumptions and will be the 
focal points of criticism and attack* 

To illustrate my point, consider an article by Arthur 
Combs, director for humanistic education at the Uni- 
versity of Florida, which appeared in a recent issue of 
'The Journal of Teacher Education*' (1972). The title 
of the article is "Some Basic Concepts for Teacher Ed- 
ucation/' I have selected this article as an example not 
because I favor the arguments it advances, but rather 
because it happened to be sticking out kitty-comer 
from a heap of similar articles on my desk. Strictly 
random access to the heap would have served the same 
purpose. Here are two assertions advanced. 

1. 'The production of an effective teacher is a 
highly personal matter, dependent primarily 
upon the development of an appropriate system 
of beliefs." (p. 286) 

2. ''Effective teacher education must concentrate its 
efforts upon meanings rather than behaviors/' 
(p. 287) 

Combs goes on to argue, "It is conceivable that requir* 
ing a teacher education program to define precisely the 
behaviors it hopes to produce may be the surest way to 
ilestroy the effectiveness of its products by concentrat- 
ing everyone's attention on the wrong dynamics'* (p. 
288). It Is clear that Combs' value orientation places 
htm in a teacher education camp far different from that 
of performance-based adherents. If a performance- 
based approach could be shown to achieve its aims, I 
would expect Combs' reaction to be, "Fine, but that's 
not what teaching is all about." 

Returning to the five assumptions stated above, it is 
pi>ssible to consider each individually and to rais^ 
questions a out its appropriateness. One could ques- 
tion the extent to which teaching is actually a science 
which can be analyzed and described in precise tenhs. 
Given current knowledge about teachers and teaching, 
one could raise questions about the ethical issues in- 
volved in identifying relevant teacher competencies and 
icVets of performance. Even if certain performances 
ye;e known to be related to teaching succes«>, the ques- 
"^ticn of whether such performances can be taught to 
prospective teachers is not at all clear. Within the con- 
text ot performance-based teacher education, such is- 
sues are tagged as *'problems to be solved," not as 
''considerations to be weighed in determining whether 
we should have performance-based programs." 

Performance-based ter.cher education advocates have 
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fviKVcdcil from a belief system wliieh aeeepis these 
five assuinpiiiMiN as piveiis. As a result, what is fvrhaps 
the nu^sl impiulant esaluatise qiiestiim abiuit perform- 
anec-baseil teaeher ediieatiiMi -whether or not it is bel- 
ter iluui eurrent praetiee.> and should be implemented 
has been answereil in many states. I he likely impael 
i>f Uiis value judgment on states, teaeher education in- 
stitutions, prospective teachers, and students outweighs 
the impact of evaluations carried out within any ongiv 
ing perfi>rmance-baseil program. Once one's belief sys- 
tem accepts the neeil lo gi> ahead with perf'>rmance- 
bascd teacher educa.ioii and certification, probl'MUs 
reduce ti> thtwe ci>ncerned with implementation and 
"spreading the gospel." It was possible to have ap- 
proached the issue of performance-based teaeher edu- 
cation as a siK-ictal experiment, emphasizing planned 
intervention and evaluation of the basic notion itself 
(C*ampbciL 196^). However, it should be noted that 
this experimental approach also proceeds from a belief 
system, one which accepts 4 rational, planned, gradual 
approach to siK'ictal change. 

' Regardless of the basis on which the decision to in- 
stitute a pjrformanec-based approach is made, how- 
ever, onc; it is agreed to commence, issues related to 
evaluutii n within ongoing programs arise. It is to these 
problems I snail now direct attention. 

Evaluation in Performance-Based Teacher 
^Education Programs 

A multitude of models which integrate both student 
and curriculum evaluation concerns of instructional 
programs have been advanced in recent years (e.g., 
Airasian, Madaus, and Rakow, 1972; Provus, 1969; 
StufHebcam, 1971). These models describe various 
types oY evaluation, the points in instruction where the 
types are most relevant, and the kinds of judgments 
which can be Inade on the basis of the evaluations. 
Mi>st models arc specifically designed to help indivi- 
dualize instruction within the context of programs 
which emphasize predefined objectives or competen- 
cies. Because such mixiels exist, I shall not attempt to 
suggest a full-blown evaluation model for perform- 
ance-based programs. Rather. I shall direct may atten- 
ium to a few evaluative issues which 1 feel are of 
major im^x^rlancc fi>r any performancHJ-based teacher 
education model. 

Defining Teacher Competencies 

While some observers -^rgue that the mi>st powerful 
individuals in a performance-based approach arc those 
who ultimately certify performance or competency, I 



would argue that the nii^st powerful individuals are 
those who frame the competencies lo be attained. 
I hese are the individuals who explicitly detine what is 
a giH>d teacht I he decisions of these individuals color 
the selcctj.i>h of learning experiences as well as the 
evaluative techniques and criteria. In the^performance- 
based approach, which proceeds from identification of 
ends to selection of means to obtain these ends, it is 
the ends which are paramoftnt. The rationale for a 
program, its learning experiences, standards, and certi- 
fication practices a-st upc^n the |xrrformances defined as 
ticcded by the good teacher. Parenthetically, there is ? 
vast literature on the pros (e.g.. Bloom. Hastings, a d 
Madaus, 1971; Blwk, 1971; Popham, 1968; Tyler, 
1934, 19S()) and eons (e.g., Atkin, 1968; Doll, 1971; 
Eisner, 1966) of an approach to education which pro- 
eeeds from clear articulation of ends to selection of 
means. Seldom dws this literature appear in the con- 
text of discussions abt^ut performance-based teacher 
education. 

It is at this first stage involving the definition of 
competencies that evaluation should play a major role. 
Just as controversy about performance-based teacher 
education generally will center upon its* assumptions, so 
will controversy about any given performance-based 
program center on its specific definition of the good 
teacher. In short, one of the primary domains of evalu- 
ation within a given program will be its goals; i,e., the 
competencies it seeks to teach. That this will inevitably* 
be the case is a function' of the fact that not everyone 
defines a good teacher in the same way. 

Wise administrators of performance-based programs 
will build evaluation of their program's defined compe- 
tencies into their planning efforts. One useful model for 
such - evaluation Ms suggested by the National Assess- 
ment of EdMC5|tional Progress study staff. The NAEP 
sought to provide nationwide census-like datat about 
the educational attainment of Americans in a variety of 
subject fields. In planning the assessment, the ^aif 
quickly came face to face with the problem of defimng 
what the educational s7Stem was trying to achieve. In 
essence the problem was to define the competencies 
possessed by ihe giuvJ student in areas ranging from 
arithmetic to citizenship* A tentative list of objectives 
was determined for each content field by subject matter 
specialists. Ihesc objectives and suggested procedures 
for gathering evidence about their mastery werti sub- 
mitted for evaluation and revision to panels composed 
of educators, concerned citizens, and scholars repre- 
senting various points of view and sections of the 
ci>untry Some of the panel sessions were heated; some 
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resulted in objectives being deleted because consensus 
regarding their appropriateness or impimance could 
not be reached. In the end, however* a general consen- 
sus was reached and a list of objectives acceptable to 
representatives of diverse segments of the populace was 
advanced in each subject area. 

Thus, once a performance-based program has gener- 
ated an initial specification of the competencies it seeks 
to instill in prospective teachers, numerous interested 
and affected publics such as parents, teachers, students, 
administrators, scholars, and others should have an op- 
portunity to judge, criticize, and revise the competen- 
cies. Not only docs such a screening process have a 
public relation value, but more importantly, it affords 
input into the process by publics who are likely to be 
vitally concerned and affected by the performance- 
based approach. It is likely that reconciling the inpu s 
and judgments of,, varied constituents will be difficult, 
time consuming, and often frustrating. However, the > 
importance of the specified competencies in directing 
all aspects of the performance-based program man- 
dates such an evaluation at this early stage, 

' Evaluating Student Progress 

Once Some degree of consensus regarding the ends 
of performance-based teacher education is reached, the 
problem of evaluating student progress through the 
program becomes relevant. There are three domains 
which call for student evaluation. The first involves^ 
knowledge of processes, theories, techniques, etc., in- 
formation which is best evaluated by paper and pencil 
means similar to those utilized in' nonperformance 
credit and course-centered approaches. The second do- 
main involves teaching practices and activities, demon- 
strations that the student can a(itually perform and uti- 
lize various strategies, modes, and techniques. Finally, 
students will be evaluated in terms of the extent to 
wh'ich their knowledge and performance capabilities re- 
sult in improved classroom learning. 

The third domain, which may be termed the product 
or output domain, will be discussed in a later section 
of this paper. In this section I will consider the knowl- 
edge and practice domains. Three evaluative issues ap- 
pear relevant to the discussion: the specification of 
entry behaviors, the mode of grading student perform- 
ance, and the problem of obtaining an adequate sample 
of student behavior. 

While performance-based advocates stress the exit. 
l)ehaviors of students who pass through their programs, 
it may be appropriate to raise some questions related 
to student entry behaviors. Note that emphasis on exist 



behaviors implies an acceptance of the pro{%sition that 
enough is known or can be known about instructional 
techniques to insure that most individuals can attain 
the prespecified competenci^ of a program. Despite 
some p^^rsonal reservations about such an assumption, 
I am willing to concede that it may be appropriate and 
true. However, one might question whether it is worth 
the time, effort, and expenditure for a particular stu- 
dent to take 8 years, to obtain ultimate initial certified- 
tion when the majority of studens^reach the same 
stage in 2 years Should, then, prospective candidates 
for performance-based programs be Evaluated with re- 
spect to their attainment of a minimum set of entry 
competencies before being accepted into the program? 

Undoubtedly some initial screening of candidates is 
always done. Candidates who do not possess adequate 
academic records or test scores are rejected. C^idates 
who do not manifest particular Interest, pe^nality, 
and value characteristics are not accepted. But evi-. 
dence about the cognitive and affective status of indi- 
viduals is typically gathered by means of global* 
imprecise, impressionistic means, the very means 
performance-based programs strive to eradicate. It 
would appear worthwhile, then, to evaluate prospective 
students in terms of a set of entry competencies which 
are specified in the sa ae manner as the exit compcten- * 
cies. Certainly it is impossible to.< identify a complete 
list of entrance competencies at< this point in time. 
However, spme entrance criteria can be identified and 
made explicit. Such criteria would require monitoring 
and revision over successive groups of students to ar- 
rive at a more complete and relevant set, but the im- 
pact of well defined cntrj^ competencies may be worth 
the effort insofar as they may reduce the frustration of 
unqualified candidates as well as serving as a means of 
placing accepted candidates into the optimum instruc- 
tional starting point (Airasian and Madaus, 1972a). 

Once students are into the instructional process, it is 
necessary to determine whether or not they have at- 
tained mastery of the required competencies. The first 
issue here is, of course, what is meant by mastery. 
That is, one must identify the standard of performance 
students must attain to be certified as possessing a 
competency. Obviously, there is no source, other than 
judgment, to which one can refer to select appropriate 
standards. The question of standards is one which 
plagues all evaluation efforts. However, the nature of 
the competency, its relevance to instruction, its sus- 
pected impact on classroom learning and other $uch 
considerations should be weighed in setting the stand- 
ard. For example, based upon consideration of the 
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above areas* I have set a mastery standard of 85 per- 
cent in my secondary level course in tests and measure- 
ments. A student is certified^ as Omastering tests and 
measurements wijj^n he demonstrates competency on 
85 percent of my cour4«5 objectives. Were I teaching 
physical education to^ secondary education ni$ijors con- 
centrating in areas other than physical education, I 
would set a lower mastery standard. At any rate, one 
must have some criterion which^enables him to d.'ter- 
mine whether a given performance is indicative of 
competency, v 

With the prior specification of competencies and 
standards, the problem of grading students is relatively 
straightforward. The crucial judgment to be made is 
whether the student has attained mastery of a compe- 
tency, not how he has perfomied relative to his peers 
(Airasian and Madaus, 1972b; Popham, 1971). As 
various authors concerned with' performance-based 
teacher . education have indicated, criterion-referenced 
as opposed to norm-referenced evaluation is needed. 
With respect to each required competency, we'will want 
to know whether the student has achieved criterion- 
level performance, not how high or low he stands rela- 
tive to other students. Criterion-referenped evaluation, 
which many argue is more relevant, hwnane, and 
**fair'' than norm-referenced evaluation, is possible only 
when criterion^performance is predefined. One of the 
major advantages of performance-based models is that 
they dp incorporate prespecified, well defined exit be- 
haviors. As an added bcni^fit, the availability of clearly 
stated exit behaviors which serve as evaluative Criteria 
enhances diagnosis of student learning difficulties. Un- 
like the course grade method of evaluating performance, 
a criterion-referenced approach related to identified 
competencies can provide the student with more pre- 
cise diagnostic direction than ''study more*' or ''work 
harder.'' In a sense, all evaluation taking place prior to 
actual certification of competence is formativ^ diag- 
nostic evaluation. \ 

Actual data gathering techniques to evaluate knowl- 
edge and practice competencies ar6 not complex. For 
knowledge competencies, paper and pencil tests, oral 
examinations, and the like are appropriate. For prac- 
tice competencies, studies* performance in classroom, 
microteaching, or other similar situations can be 
evaluated by one, or preferably more, judges on the 
basis of checklists or overall performance. The basis 
for judgment, be it judgment of knowledge or practice, 
is always performance relative to the predefined com- 
petencies. The extent to which evidence gathering situ- 
ations permit students to manifest the behaviors inher- 



ent in the competencies is the extent to which ahe 
evaluation is valid: 

One note of caUtion must be introduced^ however. 
Any testing situation provides only ix sample of a stu- 
dent's behavior. Relatively sli^t differences in test 
items or classroom situations from those utilized in an 
evaluation may engender different student performance* 
Thus, in judging competency, paiticulariy of teaching 
practice behaviors where evidence gathering anij evalu- 
ation ace more time consuming, it will be important to 
include more than a single measurement. Students 
should be required to demonstrate competence in class- 
rooms of varying types J^fore they are certified as 
competetit. 

Competencies E'^uateci 

In addition to helping judge curricular adequacy and 
student learning, evaluation plays another powerful 
role in any instructional system: it defines for thdse 
being evaluated what the important or "real^ aims c. 
the instructionalvrogram are. We have all heard stu-* 
dents remark, "Forget what he tells you in class* If you 
want an A, memorize the book.*' The implication of 
such a statemei\,t is thait the grading process — ^that 
'which ultimately counts-r-is based upon a very limited 
subset of the totality of instriictionjEd activities and 
aims. 

^he relevance *of evaluation's role in defit^ng the 
real aims of instruction for performance-based teacher 
education relates primarily to affective performance* 
No performance-based approach worth its spit wilj 
argue that teaching competence is comprisedltolely of 
cognitive and psychomotor skills. Attitudes, values, in- 
terests, preferences, and the like are at least as impor* 
tant to the competent teacher as cognitive and psy- 
chomotor ability — peiliaps more so. However, 
instructional programs based upon the clear identifies* 
tion of objectives or compeMUcies have a lengthy his-* 
tory of paying lip service to the importance of affective 
outcomes, but concentrating evaluation efforts on more 
^asily measured nonaffective areas. By following such a 
practice, evaluators define the .cognitive and psychomo- 
tor aims as the really important objectives of the. 
progrllm, regardless of what the program designers 
claim about the importance of affective aims. It doesn't 
take long for students to "psych out*' what is required 
of them and to shape their behavior to comply with 
expectations. 

Thus, if affective performances are important, they 
must be evaluated. Unquestionably the state of the art 
of affective assessment lags behind cognitive or psy- 
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chomotor assessment. In the end, interpretive judg- 
ments based upon both formal arvl informal i>bserva- 
tton? and diseussit>ns will probably provide ^h^ 
optimum means of gathering afTeetive evaluative data 
about student progress/ Fhe laek ot objectivity associ- 
ated with such techniques in wmparisi^n to more for- 
mal paper and pencil techniques should not deter eval- 
tlation. "One method of stressing -the importance of 
affective aims is to diagnose and evaluate them. 
« 

Data Maintenance 

♦ 

In the course of evaluation in performance-based 
teacher education, niuch'^data about individual students 
will be gathered and assessed. Two issups related to 
the maintenance of such data must be considered. 
First, one miist consider the form in which individual 
student data will be kept and disseminated! Second, 
one must consider to whom data wilt be disseminated. 
Each of these issUes will be considered in turn. 

Performance-based educational progiams start with 
a clear definition 6f the capabilities possessed by the 
good teacher. The eaf5abilities are specified in behav- 
ioral terms, not in terms of number of credit hours, 
grades in courses, and the like, it seems reasonable, 
^giv^n differences between performance-based and more 
traditional programs, tha^ student data wijthin perform- 
ance-l?ased approaches be maintained in a different 
manner than in terms ^ of course grades or i:redits 
earned. Rather, student records should contain an indi- 
cation of student attainment of every relevant perform- 
ance.. While the prior statement may hardly sound 
insightful -or revolutionary, recordkeeping* and dis- 
semination in practice are likely to become time- 
consuming and unwieldy. One of the few advantages df 
the letter grade or course credit recordkeeping system 
is that it pcrmitsnis to describe a student in a short, ef- 
ficient manner. An ihdex jcard sufiices to charactefize a 
student over 4 college years. In an attempt to over- 
come the lack of specification inherent in grades, and 
credits, performance-based approaches turned to^ the 
identification of specific behaviors related to good 
teaching. To characterize student learning in the li^t of 
the extensive lists of performances identified in most 
performance-based programs, pages and pages of rec- 
ords are needed for each student. It is important that 
such records be kept and disseminated in unshortened, 
unoxpiirgated form to state certification boards, pro- 
spective employers, and to the students themselves. 

Individuals responsible for recordkeeping in per- 
iormance-based approaches must resist the inevitable 
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pressures from school administrators ana certification 
hoards to provide a number or a brief statement to 
characteriw a student. To succumb to such entreaties, 
no matter how attractive they appear or strongly they 
arc urged, will result in rc^ducing the importance of the 
prespecified performances, which represent after all, 
the basic appeal of performance-based approaches. 
Further, to succumb to such entreaties will i^turn certi- 
fication aVid liiring practices to a **highest number 
wins" basis, with the result that specific relevant capa- 
bilities will be ovcrlo6k';d or ignored. 

AdeqAiate I'ecordkeeping reprijsents only one aspect 
c)f the data maintenance problem. To a much greater 
extent than more traditional approaches, performance- 
based instructional programs will have on file explicit 
affective judgments about students. While our schools 
and society accept cognitive information about individ- 
uals as a prime currency of worth or merit, jhere is 
some reluctance to judge individuals t>]l th^ basis of 
personality, value, or other affective characlerisitics. 
There is a feeling th^ these aife mofe private and le^s 
/necessary for others to know. That affective qualities 
are important to good teaching, few would argue. 
However, a crucial question to be answered within the 
context of performance-based programs is 'To whom 
should judgments about a given student's values, per- 
sonality, interests, and preferences be disseminated?'* 
Clearly some individuals and agencies will require such 
evaluative information; but which, under what circum- 
staf^^ces, in what form, with what guidelines, and^or 
hojst long are only a fe|y of the concerns which should 
•gifide dissemination, There are no right or wrong, ethi- 
cal or unethical answers to these questions. It is impor- 
tant, however, that every performance-based teacher 
education approach find answers to these questions, 
answers which proti9ct.the privacy rights of its students 
yet afford information access to those to whom It is es^ 
sential. 

Judging Teachers by Their Students' 
• Achievement 

The final evaluative issue to be raised is undoubtedly 
the most emotion-charged aspect of performance-based 
teacher education: the evaluation of teacher compe- 
tence in terms of .student learning. I suppose that in a 
real sense, my own dual responsib^ity as a professional 
educator and a parent reflects the problems involved in 
judging teachers* performance by their students' learn- 
ing. As a professional educator, I recognize the myriad 
of fac.tors outside ^he teacher's control which affect 



Icarnine. But as a parent. 1 wonder whether notMmld- 
ing teachers ocn minimal!) aeei>untable f(*r stiulent 
learniuii is an aceeptance ol the {H>siiion that the rea- 
si>n for failure reM.> upi>n the student s shoulders. 

Tlw data* which must be gathered to evaluiite teach- 
ettf' efTeets U|H)n student learning avy not at all clear. 
Personalitv and cognitive differenees between teachers 
teaching the same type of chisses as often' as not* result 
in the siimc oycrall class performance fi)r both teacher 
types. Rescurch itidicatcs tirat unsubstantial portion of 
the variance in student ability and achievement is at- 
tributable to cTivironmcntal factors whose major impact 
on ih€ student occurs before the student reaches the 
schoi>L I he effect of failure in prior years upon subse- h 

. quent performance remains ihc subject of discussion 
aipong educators, .Mthough none of u.s relish the impli- 
cations, k may be that prolonged failures In schpol ex- 
periences ultimately results in students who are simply 
not reclaimable. regardless of any teacher's competence 
or perseverance. We might even questio^ whether fail- 
ure or poor performance in some course^ is not a ben- 

^efit'to students/ especially if it prevents them from 
pursuing careers or areas where they^hav^ Uttle proba- 
bility of success. Finally, and perhaps Jmost impor- 
tantly, there is the problem of the critcria\ which indi- 
taie successful student learning. The pi>ssbilities here 

.arc mvriad, . 

In sum, while it is always possible to evaluate teach- 
ing competency by measuring student learning, the is- , 
sues remaining to be settled before such evaluation cah 
be undertaken in an intelligent manner, fair to Ijpth 
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teachers and students, suggests that student learning 
measures not be used to evaluate individual teachers at 
present. Rather, research on variables and measures 
hypothesized to relate to student achievement should be 
undertaken. Particular tea^-her competencies should 
first be validated against, measures of student behavipjr. 
Note, hi)wever. that the most accurate and extensive 
research Afata/ will not remove the ultimate need for 
making evaluative judgments concerning the type and 
form of student behavior which will be considered in- 
dicative of competent teaching. 

Conclusion 

N 

This paper has argued that the Yeal evalnation ques- 
tions inherent in performance-based teacher education 
relate not to constructing objective evidence gathering 
techniques j^^r measures, but rather to judgments about 
what data are to be gathered, from whom, and under 
what conditions. *The selection of instruments and tech- 
niques is an outgrowth of Ihesfe prior evaluative deci- 
sions. Those whose primary interests are concerned 
with how questions, need only consult most measure- 
ment 'exts or the curriculum evaluative' literatufe. If I 
have left some of my audience unsatisfied because I 
did not address more practical how to questio^, my 
sole defense is that I believe identification and resolu- 
tion of the what questions is the surest manner of .nak- 
ing performlance-biased teacher education a move via- 
ble, effective, and humane approach to the problem of 
educating teachers. 
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The State Of The Art In Performance Assessment 

Of Teaching Competence 

By 

FREDERICK J. McDONALD 



N To describe and evaluate the state of the art in per- 

> forroance assessment of teaching competence, wc 
r should answer three questions: (1) What are the goals 

of the systems to measure teaching competence? (2) 
What prog^ have we made in developing the proce- 
dures required for systems measuring teaching compe- 
tence? (3) What problems must be solved to create. 
% effective and cf!ident systems for assessing teaching 
competence? 

Before attempting^ to answer these questions, I 
should first openly acknowledge the assumptions that 
influence my ideas ^nd those that do not. I assume that 

> • even small progl^ss made in assessing teacher compe-^ 

tence will be a gi eat improvement over our present 
evaluations. Because I assume this, I^, am willing to 
4jpe the use of procedures and systems which at the 
present time are lin^ited or even defective—^since I also 
assume that as we use what instruments and techniques 
are now available we shall learn more about the nature 
of teaching competence and progressively improve our 
methods jfor evaluating it. I further assume that the 
cost of measuring Ceacher competence will iSbt neces- 
' sarily be prohibitive, despite the fact that many people 
I seem to assume the contrary. It is obvious that institut- 
ing a measurement systerh where there was none be- 
fore will impose heavy costs. But until wejfiave a com- 
plete cost/benefit analysis, I am not willing to assume 
that these costs wilt be unmanageable'. Further, I do 
not assume that the evafuation system will necessarily 
. be complex because we do not know as yet what infor- 
mation is needed to make reliable and valid judgments 
about teaching competence, i choosy, therefore, to be 
optimistic about both the cost and complexity of an 
evaluation system 

ft seems to me that the most reasonable position to 
take at this time is to acknowledge the many problems 
and to regard their solution as dependent upon Jhe or- 



dinary processes of research and development. It is 
self-defeating and destructive, to so negatively prejudge 
the situation that work which might resolve the prob- 
lei^is either impeded or prematurely terminated. 

The Goals of an Evaluation System 

Let us now return to the three questions with which 
I began this presentation. The first is obviously the 
most important: what shduld be the goals of a per* 
formance assessment system? The ultimate goal of such 
a system is to provide information that enables the de** 
cision makers — teacher educators, certification officers, 
and administrators — to decide whether a teacher candi- 
date has sufficient C9mpetence to be permitted to 
teach. Everything else that we will want to say about 
this goal is either a refinement of what is meant by 
making a judgment about competence or' a problem 
that must be solved if such judgments are to be made^ 
I will therefore list a set of conditions that must be sat- 
isfied for this goal to be attained. 

First, measurement procedures used in the evalua- 
tion of teaching competence must have hi^^h validity. 
Validity in .this case me&ns we have demonstrated a 
substantial relationship between* a teaching skill or per* 
formance and its effects upon students. 

At this^point in time we know very little about what 
skills or performances have demonstrable effects on 
student behavior; The highest priority in a research 
program on teaching competence must be given to 
solving this problem. The conclusion, however, should 
not be drawn that we must defer the development of 
an evaluation system until al! the relevant research has 
been done. We already have an abundance of ideas on 
pertinent teaching competencies which we can begin to 
measure — a necessary first step, and one whose^ effect 
on teaching performance can be studied systematically 
as part of the process of developing evaluation systems. 
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We cannot or ought not. however, make definitive 
judgmenis about teaching ci>mpetence until ihc validi- 
tres of specific teaching skills or performances have 
been established. . 

Second, we must establish the reliability of assess- 
ment procedures that will be used, in evaluation sys- 
tems. Reliability in this case means the stability of a 
teaching skill or performance. Does the teaching skill 
vary considerably from day to day, class to class, topic 
to topic? We need to know the conditions under which 
it is most likely to be variable and those under which it 
is most likely to be couNtanl. 

While considerable atiention is given to the problem 
of validity, practically no attention has been given to 
the ^ performance characteristics of teaching skills. 
There is some intuitive understanding that a variety of 
conditions may affect a teacher's .skill, but formal re- 
search H>n the problem is nonexistent. It makes no dif- 
ference how research m this aspect of teaching per- 
formance c^mes out because We need all kinds of 
information in making judgments about competence. 
If, for example, a parMcular teaching skill requires spe- 
cial conditions to elicit it, we must ask whether those 
conditions were present when the teacher's possession 
of the skill was being evaluated. 

Third, we need to know what, for lack of a better 
term, I will call the 'learnability" of a skill. It is con- 
ceivable that a particular skill, even though it has been 
demonstrated to have highly desirable effects upon 
children's learning, can be learned by only a few indi- 
viduals. We do not expect everyone to become theoiCt- 
ical physicists or concert pianists or gold medal-win- 
ning athletes because we know that some special 
aptitudes may be required for achievements of this 
order. Similarly, there may be forms of teaching whose 
acquisition demands unusual aptitudes. Thus, we 
should avoid the trap of a.ssuming that every skill dem- 
onstrated to have desirable effects on stiidents will nec- 
essarily be required of every person/ who wishes to 
teach. 

We must develop the kinds of information described 
in these three points — relating to the reliability, valid- 
ity, and learnability of teaching skills. We cannot 
achieve our goals unless we can define what is to be 
measured, under what conditions it is to be measured, 
and how the information yielded by the procedure is to 
be interpreted. But there are some other conditions be- 
yond these three that ma^t be met by any system for 
measuring teaching competence. 

A fourth condition, for example, is that the informa- 
tion gathered about teachers must be as uncontami- 



nated by subjective biases and political pnKcsses as is 
humanly piLWsible. Any observation of human beh iNior 
is, or course, inlluenccd to a degree by the characteris- 
tics of the observer and by the characteristics of the 
measurement procedure itself. We shall need to de- 
velop new strategies for collecting information on 
teaching competence if we are to minimize the prob- 
lems of gathering such information. 

Fifth, the conditions of measurement must provide 
comparable information on groups of teachers. We 
must standardize the conditions under which teacher 
behavior is measured. Otherwise, it is impossible to tell 
whether variations in teacher behavior are due to situa- 
tional differences or to differences in teacher compe- 
tence, thus exposing the candidate to the vagaries ot 
idiosyncratic favoritism. 

The conclusion should not be drawn, however, that 
the assessment system should Jgnore variations in the 
conditions under which teacher performance occurs. As 
I stated earlier, it is essential that we determine those 
conditions under which a teaching skill may be ex- 
pected to vary. Such information is necessary in evalu- 
ating teaching competence. But we cannot treat teach- 
ing as if it were so different on each separate occasion 
that we can never evaluate it. The conflict between es- 
tablishing reasonable expectations for teaching per- 
formance and the variety and complexity of the situa- 
tions in which teaching occurs is one of the most 
important problems we have to solve. Until it is solved, 
our decisions about competence must necessarily be 
tentative. 

Sixth, we must develop an evaluation system that 
makes due allowance for the teachers* opportunities to 
acquire the relevant teaching skills. A decade may well 
pass before teacher education programs have bec<mie 
performance-based and before a reasonable amount of 
performance assessment has taken place. To demand 
competence of individuals who have not had the op- 
portunitites to acquire that competence is immoral, and 
to make judgments that affect their careers on such an 
insubstantial basis is equally immoral. Assessment 
should not be used for evaluative purposes when the 
individuals being assessed have not had the opportunity 
to learn those things for which they are being evalu- 
ated. 

In listing these constraints, I have outlined the major 
problems that must be solved if a system of assessing 
teaching competence is to be developed and used in a 
practical way; indeed the solution of the problems de- 
pends on our meeting these conditions. 



ERIC 



BEST COPY AVAIUBLE 



The State of Measurement Technology 
and Knowledge ^ 

Despite popular opinion to the contrary, measure** 
ment technologj- follows the wants of Its users. Fior 
three-quarters of a century, decision makers of one kind' 
or another have wanted to assess what candidates for 
teaching positions knu\\\ Measurement technology for 
assessing academic knowledge thus became highly de- 
veloped. Consequently, we now have widely available 
tests of knowledge of subject matter and of knowledge 
about teaching methods. But until recent years there 
has been no demand of any consequence for the devel-" 
opment of performance assessment. As a consequence, 
this aspect.of measurement is largely undeveloped. 

We have developed observation scales which permit 
us to report what teachers do under ordinary teaching 
conditions; but none of these observation schedules or 
methods meet any of the constraints which 1 outlined 
previously. The conclusion should not, however, be 
drawn that the use of observation schedules should be 
terminated: nevertheless, they and any other measure* 
ment procedure must be subjected to the kinds of re- 
search and development necessary to make them useful 
within the stipulated constraints. 

The basic problem of performance assessment tech* 
nology is to create a controlled situation or to use an 
ordinary teaching situation with sufficient knowledge of 
its characteristics so that their influence on the per- 
formance can be assessed. Observation schedules fall 
into this secorjd group of procedures. The problem of 
assessing the influences in a wide variety of teaching 
conditions and Iperformances is enormous. I have come 
to the conclusion that a more profitable line of re- 
•i*- search should use controls, simulated teaching situa- 
tions. \ 

Rather than! describe the adequacy of available 
measurement technology, a rather depressing picture, I 
shall describe the work in progress on the assessment 
of performance; using simulated teaching conditions. 
This work is r<ilatively new in the field of teaching 
competence, and it is conceivable that it may never de- 
velop into a practicable technology. But some of the 
problems must Ipe solved and are more likely to be 
solved, efficiently, and satisfactorily by using simulated 
teaching conditions. The relationships between teaching 
performance and student learning, for example, can be 
studied very efliciently by using microteaching sessions. 
Similarly, the reliability of teaching performances 
under controlled conditions can also be easily assessed. 

We are attempting to build a system that provides 
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information about basic teaching skills, teaching per* 
formances, and teaching strategies. This system is de« 
signed to use simulated teaching situations of varying 
degrees of complexity. Initially, both the teaching skilt 
measured and the conditions under which it is meas- 
ured are relatively simple. The prospective teacher 
moves through th.e system from this simple initial level 
to one where he or she must use a variety of skills, 
performances, and strate^es in combination and must 
enact them over a period of time. The critical feature 
of the component systems is that the conditions are 
carefully controlled. In what ways are they controlled? 
For any given component, the teaching objectives and 
teaching performances are specified; for example, if we 
are attempting to assess the teacher's ability to rein- 
force students participating in a discussion, the objec- 
tive of the lesson Is set as increasing the amount of 
student participation in the class. The trainee is told 
that he will be evaluated on the degree to which stu- 
dents participate in a class discussion. Initially, we 
want to measure only his use of reinforcement proce- 
dures to facilitate participation in classroom discussion. 
In other components and at other times, the teaching 
situation will be constructed to evaluate the trainee's 
ability to present a stimulating problem for discussion 
or to ask questions which elicit comments from stu- 
dents or facilitate discussion by highlighting differences 
of opinion. 

In order to make assessments at this time, certain 
factors have to be controlled. For example, if we are to 
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study the teacher's use of reinforcement techniques, we 
do not want the teacher preoccupied with the prepara- 
tion or development of the topic or with the choice of 
the topic, nor do we want a group of teachers to have 
different objectives. Therefore, as I noted, we stipulate 
the objective, and we also provide the teaching mate* 
rials for the lesson. Further, we control variability of 
the students by reassigning the students to a teaching 
assignment and by providing the preparation period 
for familiarizing themselves with the material to be dis- 
cussed. It should be noted, and I wish to emphasise, 
that the factors which are controlled in one type of sit- 
uation may be systematically varied in another in order 
to assess the degree of the teacher*s skill. 

What can be learned by using this technology? 
These mibroteaching sessions can be used to assess a 
teacher*s slqll under a variety of different teaching con- 
ditions. We wave tried several formats of microteaching 
lessons to determine what can be learned by having 
teachers teacn different lessons and different groups of 
students. In one format, for example, we asked the 
teacher to teach four different topics to the same group 
of students; in another, a teacher teaches different stu- 
dents the same topic; and in yet another, the teacl)^r 
will teach different topics to the same students and 
then the same topics to another set of students. In 
these ways we can test how the teacher's skills change 
under the different teaching c<Hiditions. We have al- 
ready found that inexperienced teachers* behavior var- 
ies from topic to topic, to some extent, even though 
the topics appear to be highly similar in natme and 
would seem to offer similar opportunities for using 
teaching skills. 

Eventually, we would. expect to have a series of rep- 
resentative teaching situations in which it had been 
shown that certain kinds of teaching skills are highly 
likely to be useful. An assessment battery would con- 
sist of a series of microteaching sessions covering these 
teaching situations. 

Because the microteaching format has several limita- 
tions, such as relatively short lessons and small num- 
bers of students used, we developed a mini-course 
format to use for more complex teaching situations, A 
mini-course in this context is a short course lasting 1 
or 2 weeks. The teachers teach this course an hour a 
day for one of these periods of time. For these courses 
we have also developed a complete statement of objec- 
tives, course materials, and achievement tests to be 
taken by the students at the end of the course. Almost 
300 preservice teachers have taught these courses so 
that we have gathered considerable data on teaching 



performance and experience with the technology itself . 

The results of the analyses of these teaching per- 
formances indicate that the microteaching perform- 
ances are relatively poor predictofs of the teaching per- 
formances in the mini-courses. But, for several reasons, 
1 am not willing to offer this conclusion as solidly sub- 
stantiated. First, the subject-matter of the shon courses 
is more complex, and segments of any one lesson 
would be similar to the teaching situations created by 
the microteaching session. But their combination pre- 
sents a different kind of teaching problem. Also, in the 
situation in which we have gathered data, relatively lit- 
* tie performance training took place bei een the initial 
microteaching and the subsequent teaching in the short 
courses. Systematic training using the assessment gath- 
ered in the microteaching might influence substantially 
the relationships between the two kinds of perform- 
ances; that is, where the teacher has sufficient skills, as 
revealed in the microteaching performance, the instruc- 
tors would reinforce these skills so that they were more 
likely to be used by trainees in the more complex 
teaching. There is still reason to believe that we have 
created two sufficiently distinctive types of performance 
situations which will yield equally valuable but different 
kinds of information. My own opinion is that the mi- 
croteaching is more useful for assessing the degree to 
which a teacher has basic skills; the mini-course is most 
useful in assessing how teachers integrate these skills 
into complex teaching performances. 

As I said eariier, oift goal is to build a system that 
will assess the simple components of teaching compe- 
tence and also its more complex aspects. We visualize 
the coordinated assessment system as made up of pe- 
riods of microteaching used primarily for assessment 
purposes and short courses or units that can later be 
embedded in internship and student-teaching experi- 
ences. These components would be arranged so that 
the training institution could be ^ven information con- 
tinuously about the level of competence being reached 
by the trainee. 

The student teaching and internship experiences can 
be used to assess daily performance under uncontrolled 
conditions. They are useful for providing information 
on what teachers are likely to do in contrast to what 
they are able to do. The simulation-type teaching situa- 
tions that I have described should be thought of as 
teachers assessing what teachers are capable of doing; 
whereas on-the-job performance tells us what he or she 
regularly does. We can also use on-the-job observation 
to assess such factors as teaching style. 

I am convinced, after a year's experience with ad- 
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ministering systems such as these and another yearns 
analysis of the data gathered, that it is entirely feasible 
to build a useful pcrforauuice assessment system that 
can he embedded in any teacher-training program. As 
such systems are developed, state departments of edu- 
cation can learn what criteria ought to be applied to 
the assessment of teaching competence • The prototype 
that wc are developing in our research program can be 
used to generate concepts about the appropriate means 
to evaluate each training institution's procedures for as- 
sessing the competence of their students. 

It is obvious to me that the type of techniques being 
developed can be used with beginning teachers prior to 
the granting of tenure, and for periodic reassessment of 
teaching skill. We should not, however, use these pro- 
cedures at the present time for continuing certification 
schemes, any more than we ought to use the current 
observational procedures and administrator evaluations 
for that purpose; but there is good reason to believe 
that, in time, assessment strategies can be developed 
which will be useful both in preservice and inservice 
assessment of teaching competence. 

One question asked frequently is, "How expensive 
v'ill performance assessment be?'* As I observed ear- 
lier, tHjre is no adequate way not to answer this ques- 
tion because we do not know what information we 
shall need to make judgments about teaching compe- 
tence. My experience indicates that we can gather large 
amounts of data about teaching performance relatively 
inexpensively: for example, three 20-minute micro- 
teaching sessions yield an hour of teaching perform- 
ances. A videotape of these performances can be ob- 
served to assess many different skills. In fact, the short 
course yields almost more information than can be 
quickly processed. Wc have fesorted to audiotaping 
and videotaping about a half hour of each hour session 
for the very practical reason that this half hour covers 
three-fourths of the: teaching that occurs. I am not con- 
vinced that even this much information is needed'. We 
are studying this problem by taking different samples 
of the videotape performances that we have and find- 
ing the intcrcorrelations among teaching performances 
across the samples. Thi> technical work will eventually 
help us find the optimum amount of observation time 
that wc must use in ordei to make reliable judgments 
about teaching skills. 

While many people are concerned about the cost of 
performance testing, few discuss' the economies that 
may be achieved by using them. It is very apparent 
that, when one looks at videotapes of teaching per- 
formances, a diagnosis of basic teaching skills can be 



made very easily, and this information can be used to 
speed the learning of these teaching skills. We already 
have sutficieni information on feedback and modeling 
techniques to know that the information generated in a 
performance assessment system can be incorporated 
into feedback and. modeling systems to increase the 
etTeclivencss o^ iraining. Thus, we may expect the insti- 
tution to benefit by achieving economies in training 
time and the trainee to benefit by achieving compe- 
tence sooner when he enters upon actual teaching. 

Reactive Simulation Devices 

t have been describing performance assessment pro- 
cedures which require the teacher to interact with stu- 
dents in a simulation of actual teaching. There are 
some other procedures that appear to be promising in 
assessing potential teaching competence. We have de- 
veloped a filmed simulation test that portrays a teacher 
conducting a class. The film is stopped periodically and 
the viewer is asked to say what he or she would do in 
this situation; in other parts of the test the viewer is 
asked to explain what is occurring in the class, and, in 
some places, he is asked what advice or suggestions he 
would give to the teacher. We do not yet have data on 
the predictive validity of this measurement, but it ob- 
viously taps the prospective teacher^s attitudes toward 
students, his perceptions of the problems of teaching, 
and his understanding of some phases of teaching strat- 
egies. The test situation is sufficiently lifelike to create 
great interest on the part of those who view jthe film, 
and invariably the film produces some intense discus- 
sions about what the teaching portrays, thus suggesting 
to us that the film also has great potential for training 
purposes. 

Another type of. simulation device has been created 
to depict the kinds of strategies teachers use. This pro- 
cedure is a gamelike device in which an experimenter 
plays the role of student in a highly controlled way. 
The teacher arranges the subject matter in the form of 
presentations or questions, and the experimenter re- 
sponds whenever the teacher asks a question. This 
gamelike situation does discriminate sharply between 
deductive and inductive teaching styles. It seems likely 
that the role of the experimenter can be computerized 
so that the game can be used by large numbers of 
trainees in a more efficient manner. 

Both of these techniques are illusjtrative of the kinds 
that are likely to be developed to assess teacher com- 
petence. Obviously, their validity in predicting teaching 
performance has to be established. But the work to 
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date suggests that theyv arc probably of sutVicient valid- 
ity to be useful in assessing what a teacher is likely to 
do in a classroom. 

The Problems To Be Solved 

The work that I have described here is meant to be 
iUusirative of the possibilities of assessing teaching 
competence. I emphasized that the problems arc pri- 
marily technical and partially theoretical. We^Tiavc not 
yet developed an adequate technology of performance 
assessment, but the reason for this is related more to 
the lack pf experience with these technologies than to a 
lack of knowledge of what to do. 

The theoretical problems arc more complex and will 
require extensive research and development. The kinds 
of procedures that I have been talking about here can 
be adapted to experiments — for example, assessing the 
relationship of teaching skills to pupil learning. Care- 
fully controlled experiments can be designed in the mi- 
croteaching format to study hypotheses about these re- 
lationships. The mini-course data can be used to 
induce hypotheses about the relationship of teaching 
competence to pupil learning. But until the basic ques- 
tions about what competencies make n difference have 
been resolved, the design of assessment technology will 
be incomplete. 

Each of the conditions governing ian assessment sys- 
tem, thai I outlined previously, presents a technical 
problem to be solved. These problems are highly ame- 
nable to research, and there is no reason to believe 
that they cannot be solved in a relatively short period 
of time. 

A basic problem is political in character. Many indi- 
viduals believe that competence cannot be measufed 
because they do not want competence to be measured. 
The solution to this problem is obviously complex. 
Certainly the development of efficient and demonstr^a 
bly useful assessment systpms will weaken the -argu- 
ments against assessment. Still, there will probably al- 
ways be those who will say that the most important 
aspects of teaching are unmeasurable. But 1, for one, 
have never been able to understand how this conclu- 
sion can be reached before anybody has tried seriously 
to measure the more elusive aspects of teaching. 

Conclusion 

What should state departments of education, who 
wish to move to performance-based certification, do 
about the assessment problem? Obviously, an enor- 
mous amount of research and development work re- 



mains to be done, and even though we are optimistic 
about the pace at which the research and development 
can prweed, we cannot at this time offer state depart- 
ment personnel finished performance assessment sys- 
tems. For that matter, we all know that just selecting 
the competencies that will be the criterion perform- 
ances cannot be dortt at the present time. 

1 have the impression that state department person- 
nel are largely concerned, and justifiably so, about this 
latter question. 1 propose that rather than think about 
assessment in terms of what competencies should be 
measured, state department personnel think about the 
criteria for evaluating assessment systems from which 
Judgments about competence are made. If you agree 
with me that it will be several years before we have a 
good body of research established on the relationship 
of teaching competence to pupil learning, then it seems 
to me that the most practical course to take is to set 
standards for the^evidence about competence that is to 
be provided, however that competence may be defined. 

I urge your consideration of thi*: approach because I 
believe that once a faculty in an institution bec(xnes 
concerned with the assessment problem, the definition 
of competence and the demonstration of its relevance 
to student learning will become major concerns of the 
faculty and will more likely be tackled quickly and ef- 
ficiently. But it also seems to me that what state de- 
partment personnel ought to be asking is what the 
bases are to be on judgments about competence are 
made. If attention is focused on these decision-making 
processes and the information is used to make decisions 
about competence, we shall at least have guaranteed 
that the judgments are not being made subjectively or 
that a multiplicity of factors affecting competence have 
been inadequately controlled or that the range of a 
teacher*s skill has not been appropriately evaluated. 
The problem for the next several years can be stated 
as a questjon addressed to the training institution, , 
'*Given your conception of what constitutes compe- 
tence, what evidence have you gathered thht demon- 
strates that teachers have acquired tifese cbmpeten- 
ciesr 

In my opinion, if state departments would adopt 
criteria for the evidence to be presented to support 
teacher certification, the performance-based movement 
would move forward quickly. You would ^also avoid 
the trap of falling back into approving programs which 
have a performance base but cannot produce evidence ^ 
that the trainees who go through the program are in 
fact competent teachers. This approach woujd focus 
the institution's attention on the development of the as- 
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sessment of competence and would give each institu* 
tion greater freedom in the development of programs. 

Those of U5 who arc interested in the performance- 
based movement in teacher education accept a basic 
principle: we do not care how a teacher becomes com- 
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petent pvovidod that he or she can demonstrate that 
competence. If we believe in this principle, as I know 
wo do, then our efforts should be directed to deciding 
what evidence we will assess to determine whether a 
teacher is competent. 
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PBTE: Proceed With Caution 

By 

BARAK ROSENSHINE 



As ^ review the current state of knowledge about 
PBTE. ' am overwhelmed by the certainty expressed 
by state bgislatures and state departments of education 
when thcj mandate teacher performance criteria as an 
answer to educational problems. It is equally over- 
whelming to read the lists of behavioral competencies 
developed by teacher educators or to read the trainmg 
packages developed to instill' these competencies. Do 
these educators and legislators know something that I 
don't know, and why hasn't this message gotten 
through to me? 

When I look at research on teaching, I am over- 
whelmed with uncertainties. This is because systematic, 
cumulative research on teaching behaviors and student 
leam »• - barely begun. The results of the research, 
to date, art' best seen as providing suggestions for fu- 
ture research, not future practice. Research to date 
tells us very little on effective questioning skill*!, for ex- 
ample, and even less about whether effective qu'istion- 
ing skills are similar across grade levels and s\tbject 
a'eas. r 

Until informed otherwise, I assert that it is illusion- 
ary to mandate teacher performance criteria and expect 
that pupils will benefit from teachers achieving a set 
criteria on these skills. We simply know too little, and 
our research is too contradictory to support such man- 
dates. Therefore, I would caution leaders in PBTE to 
proceed slowly and with caution. Teaching competen- 
cies are not matters that can be decided by a poll of 
concerned citizens or be legislated by state legislators. 

At the same time, the interest generated by PBTE, 
and the training which is being provided under PBTE 
programs both provide settings and possibilities to con- 
duct research to answer some of the questions which 
are becoming increasingly important. The purpose of 
this paper, then, is to attempt to specify research prob- 
lems and suggest ways in which such research can be 
conducted. 



Current Knowledge on Teaching Competencies 

Anyone who writes about teaching competencies is 
aware of the lack of research which links teaching be- 
haviors to beneficial effects upon pupils. My own view 
is that it is premature to use this research to derive im- 
plications for practice or teacher training. This is be- 
cause there have been relatively few studies of teaching 
behavior and studerit learning, and most of these stud- 
ies were done on a small budget with a small sample ^ 
of teachers and a small number of observations and 
outcome measures. At present, the conclusions are not 
ready for translation into teacher training competen- 
cies. At present, we have only the faith that increased 
research, and improved research will yield knowledge 
that can be translated into teacher competencies. 

The following is my encapsulated view on the state 
of research on teaching competencies and student 
growth: 

a. The research ba.se for building teaching 
competencies is < extremely thin, because there 
have been only k small numiber of studies which 
attempted to relate . teaching behaviors to pupil 
learning and very few reviews of these studies. 
For example, although one extensive review was 
written on the correlations between teaching be- 
haviors and pupils' cognitive gain (Rosenshme, 
1971), similar reviews apparently l\gfve not ap- 
peared on the correlations between teaching be- 
haviors and pupil social or cognitive growth. Re- 
views on experimental studies on teaching 
behaviors and pupil learning have yet to appear. 

Because of the lack of research cited above 
and the unevenness of the quality of research 
which has been done, the results of the best oi 
these studies are not sufficiently clear to be 
translated into performance modules 

b. The correlational studies which have been com- 
pleted and reviewed (see Rosenshine, 1971; Ro- 
senshine and Furst, 1971) are best seen as pro- 
viding ideas for future research, not future 
practice. In each of the reviews which I authored 
or coauthored there is no section on implications 
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for practice. Rather, most of the space is de- 
voted to ideas which might be considexed in fu- 
ture research studies. In the most comprehensive 
of these reviews (Rosenshine. 1971) the presen- 
tatiim of the resuifs on each variable is followed 
by suggestions for future research. These sugges- 
tions are made not only for results which were 
consistently statistically significant or consistent 
in trend, but also for results which are, to date, 
nonsignificant. Thus, investigators who wish to 
build upon current research will have no trouble 
finding suggestions for such work. Whether these 
suggestions have merit is still to be decided. 

c. Although a variety of teacher-training skills 
packages have been produced by educational 
laboratories, research and development centers, 
private industry, and private individuals, few, if 
any of these packages are accompanied bjfftech- 
nical reports which show that training twchers 
in these skills (or a combination of skills) re- 
suits in greater student learning. 

The above paragraph is particularly distressing 
when one learns that many of these training ma- 
terials have been available for 5 years or more, 
and thousands of preservice and inservice teach- 
ers have received training in these skills. Yet, 
when one looks for reports on the effectiveness 
of such training for helping pupils growth, there 
is a lack' of reports. Those few reports which are 



Performance-based teacher education' offers 'an ex- 
cellent opportunity for colleges of education to aid in 
expanding and clarifying our knowledge base about 
teaching competencies and student growth. A critical 
question, however, is whether there, will be an exten- 
sive research effort or whether the resources V|li be 
devoted to development and a vague promise foV re- 
search at some future timfe. 

This section is written in the hopes of facilitating^ 
such research. It begins with a listing of existing 
teacher-train ng materials, raises issues about the selec- 
tion dfi^tS!|:h:ng competencies, discusses those agencies 
which might ccnducl such research, and concludes with 
a few suggestiotis on research settings and outcome 
measures. 

Sources for Teacher Training Products 

As I survey current work in PBTEI it seems that a 
great deal of time is being given to developing training 
materials. I question whether this time is necessary. In 



available were usually dissertations which fo* 
cused upon 15 to 20 teachers in one location. 

^his lack of research on products developed 
by agencies other than colleges of education is 
distressing for a number of reasons. First, it per- 
petuated a familiar educational problem of de^^ 
velop and disseminate and let validity lie in the 
eye of the beholden If well-funded organizations 
have not adequatelv tested theif procuicts, what 
can be expected from colleges of education? 
Second, the lack of research on pupil growth 
deprives the developer of vital information whicli 
can be used in modifyine the product or advising 
how the product can best be used. Third, it 
leaves colleges of education in a quandry when 
they have to select appropriate training pro- ^ 
grams. 

d. TTie lack of research in the past makes it very 
difficult to design research for the future because 
we have little to build on. Currently, when ex* 
perimental studies are conducted and nonsignifi- 
cant results are obtained, there are any number 
of explanations: the design may have betn inap- 
propriate, the tests may have been jnadequat.;, 
the treatment itself was not very important, or 
any of a number of additional possibilities. With- 
out a research base to which we can refer, we 
can barely ^egin to guess which alternative ex- 
planation most likely. 



my opinion, we already have a large number of deveU 
oped products which can be used either in present 
form or with slight modification in PBTE. Seme of 
these products can be identified fairly easily, and cur- 
rent work will identify even more. 

A major source for identifying training products is 
''Resources for Performance-Based Education'* (Hous- 
ton, 1973),. a joint product of the New York State Ed- 
ucatk)n Department and the Multi-State Consortium on 
PBTE. The book is the most comprehensive c(Hnpila«» 
tion of materials I have seen. It uicludes over 3,000 
teacher-training products which were developed by pri- 
vate individuals, universities, publishing houses, and 
government-sponsored organizations. Another source 
of teacher training materials is the third edition of the 
CEDaR Catalogue (1972). This two-volume edition 
contains descriptions of products which have been de- 
veloped by the federally sponsored educational labora- 
tories and research and development centers. It also 
contains descriptions of products under development. 



Will Research Take Place in PBTE? 
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and elaborate cross-references of the products. The 
CEDaR Catalogue, however, is currently limited to the 
products of the 20 organzations in the Council for Ed- 
ucational Development and Research (CEDaR). In 
addition, the Program on Teaching Effectiveness at the 
Stanford Research and Development Center has been 
collecting information on teacher-training products de- 
veloped by all sources and a publication of their find- 
ings should be available in late 1973. 
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If such a large number of teacher-training products 
are already available it would seem unwise to develop 
still more. A more efficient ^rategy would be to use or 
modtfjMhase to meet institutional needs. An example 
of such a program is the one developed at Florida In- 
ternational University (Spbol, 1973) in which all of 
the modules repr^ent modifications of existing, acces- 
sible material. 

Because there already exist a large number of 
teacher-training products, and because we have little 
information on the effectiveness of these products for 
enhancing pupil growth, it seems reasonable to advo- 
cate that research on existing products take a hig|ier 
priority than the development of still more unvalidated 
products. 

Yet, the evidence to date strongly suggests that still 
more development is outpacing research in PBTE. This 
is not surprising because most educational develop- 
ments of the last decade have been characterized by 
"implement and develop on a large scale and then do 



research." Unfortunately, the research is seldom done, 
and by the time we are ready to do serious research, 
the developers and their public have become occupied 
with still another "new" but unvalidated innovation 
and return to more development with another promise 
of research at some future time. 

Issues in Selecting Training Products 

The availability of a large number of different 
teacher-training products (there appear to be over 
1,000 modules in "Resources for Performance-Based 
Education," and a larger number of audio tapes, pro- 
gramed texts, kits, games, and guiaes) makes the prob- 
lem of selection extremely diflUcult. Some questions and 
issues in selection might be framed as follows: 

1. How does one select teacher competencies? 

2. How does one distinguish between competencies 
which are useful and those which are trivial and 
misleading? 

3. Are these competencies appropriate tor all sub- 
ject areas, all age levels, and all types of stu- 
dents? If not, how does one decide the most ap- 
propriate competencies for the particular setting? 

4. Which competencies are most appropriate for 
different contexts (e.g., types of student, differ- 
ing school environments, differing instructional 
materials* differing grade levels, and geographical 
areas)? 

5. Are the same competencies useful for cognitive, 
social, and emotional growth in pupils? If not, 
how does one decide the most appropriate blend? 

The common practice for answering these questions 
has been for people regarded as knowledgeable in the 
field to use their experience and knowledge to make a 
best guess. Because there is a lack of research, the 
guesses represent the best thing we have. The critical, 
question then, is whether there will be future, system- 
atic, cumulative efforts to attempt to answer the above 
issues. 

Who Will Do the Research? 

One obvious locus for research is those institutions 
doing the development, particularly the educational 
laboratories. My guess would be that if educaticmal 
laboratories are also involved in research on the effec- 
tiveness of their teacher-training products upon pupfls, 
the results of this research would feed back and im- 
prove the development of present and future materials. 

But the major place for research on the teacher- 
training performance-based packages would be in those 
institutions which are preparing teachers. Those institu- 



/■ 

/ tions preparing 500 to 1,000 teachers a year would 
have more than a cursory interest in the effects of their 
offerings, not only upon their trainees, but upon the 
pupils which the trainees will be teaching. 

Collaborative research would seem appropriate, par- 
ticularly, within a state or across a national organiza- 
tion such as the l eiicher Corps. Within the pages of 
the PBTE newsletter there are a least two examples of 
proposed collaborative research. In one, involving 
Teacher Corps trainees, a study is planned for the 
1973-74 schtH>l year involving 400 interns in 20 insti- 
tutions. In this study, approximately 10 measures of 
cognitive, emotional, and social growth of children 
taught by these interns will be collected in the fall and 
again in tht? spring. For each intern, information will 
be collected on the extent to which they possess the 
skills, attitudes, and knowledge that are believed to fa- 
cilitate the learning and growth of minority group and 
low-income elementary school children. A major re- 
search question will be the relationship of intern skills, 
attitud^, and knowledge to the learning and growth of 
their pupils. 

Another prognm described in the PBTE newsletter 
presents a two-step process-. Firsts statewide institutions 
are to implement a PBTE program which includes as- 
sessment of whether trainees achieve desired skills, 
knowledge, and attitudes. In the second step, the insti- 
tutions are required to. demonstrate whether the desired 
skills, knowledge, and attitudes are appropriate. It is 
expected that both steps will be operative before 1980. 

Funding PBTE Research 

The funds for the proposed research can come from 
many sources. One source should certainly be those 
legislatures which are advocating teacher certification 
based upon performance. Another source could be 
those educational institutions which plan to develop 
^ performance-based training modules. Even if modules 
wcren*t available, one would expect that for every $2 
spent on development and dissemination, $1 would be 
spent on research to validate" the training behaviors 
again>t desirable educational outcomes. Another source 
could be the Federal Govenment, which, to my mind, 
has not been allocating a sufficient proportion of funds 
for research into the developments they are funding. 

Settings for Researcii on Perlormance- 
Based SIcills 

The appropriate settings for research on perform- 
ance-based skills is a tricky question which probably 
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will not be answerable until a good deal of research 
has been done. The major issue is the extent to which 
results obtained in special settings are generalizable to 
regular school setting?. Special settings have been used 
in a number of studies, and these range from I(h-ni!n- 
ute Iesst>ns which student teachers present to specially 
assembled groups of pupils to unique 2-week units 
which regular teachers present to their regular pupils. 
The differences in settings, instructional materials, and 
outcome measures are such that research will be 
needed to determine the extent to which results ob- 
tained in one setting are generalizable to another. » 

At present, a case can be made for the utility of all 
settings. Small controlled settings offer possibilities" for 
intensive study, and findings obtained in tjiese settings 
can be verified in more natural situations. Thus, al- 
though teacher-training institutions would have diflB- 
culty conducting research in regular classrooms, they 
could conduct research by assembling smaller groups 
of pupils ahd noting the effects of trained and un; 
trained preservice teachers who taught mini-lessons 
lasting from 10 minutes to five I -hour lessons. 
_ To date, the number of such studies which have 
taken place on the preservice level is extremely small 
— less than one per state. It is hoped that colleges of 
education will recognize the advantage of combining- 
the implementation of PBTE with research on the per- 
formance-based skills. 

Matching Outcome Measures to Training 

One of the biggest problems in conducting research 
on teaching, whatever the setting or the performance 
skills, is selecting the approjiriate criterion . measures. 
In many studies, the evidence, .is limited to whether 
trainees modified their instructional behavior in desired 
ways. Research which contaiins trainee outcomes is im- 
portant, but it cannot substitute for research on whether 
these trainee outcomes increase pupillearning. 

If someone is conducting validation research, it pres- 
ently seems extremely important to spend a Aeat deal 
of time inspecting the match bfettjpen the tether per- 
formance skills . and the eriteriori measures. Such 
inspection and matching can take place tvvo ways. One 
way would' be to start with specific performance skills 
(e.g., asking higher order 'questions) ahd make rea- 
soned guesses on the kind of pupil learning one. can 
reasonabl;^ expect if these specific skills Ve injple^ . 
mented. (One might also c6nsider the student lealtiing 
which would not be expected.) Then one coiild foca^e 
or develop appropriate learning measures. ' 
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A second priKcdurc would be to select pupil out- 
comes (e.g., siKial or emotional growth), iaspcct the 
available measures in this urea, and then develop a ra- 
tionale for the performance skills^which seem appropri- 
ate for enhancing such pupil learning. This second pro- 
cedure has been used to reanalyze correlational and 
experimental studies in wh'^^h cognitive measures are 
the outcome. The current evidence suggests a fairly 
*high specificity of effect. That is, there is a strong cor- 
respt>ndeiice between the cognitive activites emphasized 
during instruction and those asses.sed on the cognitive 
outcome measure. If further analyses continue to sup- 
port specificity of effect for cognitive measures, such 
specificity may also hold for af» "vc outcomes. If so, 
then it would seem wise' to inspect the affective out- 
come measures used in a validation study to sec if 
there is sufficient correspondence between the affective 
^ettaviors stressed in the instructional modules and 
those assessed on th'te outcome measures. 

/Currently; we spend relatively little time inspecting 
tne match between the performance skills- and the out- 
come measures. Rather, the more common practice has 
been to select performance skills from a variety of 
sources, train teachers in these skills, and run some cx- 
beriment to see' whether the pupils of the trained 
teachers show greater learning than the pupils of the 
luntrained teachers. The usual result has been no signif- 
jicant differences. 

I (The reader who is interested in further study of the 
{design of validation studies might consult Flanders 
j (1970, chapter 12), Flanders (1971), and Rosenshine 
f and Furst (1973).) ' ' 

Irrefutable Hypotheses 

' *^ One hopes that when validation research occurs in 
PBTE, investigators and reviewcrs will' allow for the 
possibility that certain pet ideas will not be validated. 
Unfortunately, there h.»s been a tendency in education 
to hang on to all ideas, no matter what the research Re- 



sults. When faced with studies which do not support 
our pet ideas, we frequently claim that if the studies 
had been better designed, or if the tieatment had lasted 
longer, then our ideas would have been validated- 
When all else fails, we invoke The Educators' Creed: 

These are things that tests can't measure 
These may be the most important things of all 
And the experimental group did best on these. 

When training teachers in performance skills does 
not yield differences in pupil growth there are any num- 
ber of possible explanations, and these explanations are 
sources for future research. It seems unreasonable and 
unpro'' »clivc to develop lines of argument which allow 
ns to.u^isert the validity of all performance skills which 
we consider important no matter what empirical results 
are obtained. Indeed, given the large number of teach- 
ing skills which have generated, one would hope that 
research would help usy/educe the number of skills 
currently coasideied important. 

Summary - 

In performance-based teacher education (as in other 
areas of education such as the teaching of reading), we 
arc faced with a large number of training and instruc- 
tional materials, a great deal of interest, and a limited 
amount of research. As such, PBTE appears no better 
and no worse than the rest of the educational field. 

Whether PBTE will be limited to development and 
dissemination, or whether it will also incliide necessary 
research on pupil learning is a critical question which 
will be answered by actions in the next few years. The 
teacher-training materials which have been developed 
offer excellent opportunities for experimental research 
(as contrasted to the correlational research whiclf has 
dominated the field to date,) One hopes that institu- 
tions engaged in training teachers will cooperate ih de- 
veloping a systematic and cumulative knowledge base. 



33 



References 



CEDaR Catalogue (3d edition). Commercial-Educa- 
tional Distributing Services, P.O. Box 3711, 
■ Portland. Oregon. 1972. 

. * ♦ 

Flanders, N. A. "Analyzing Teaching Behavior." Sos- 
ton? Addison- Wesley, 1970. 

Fiandoe^ N. A. A national coordinated progranr of re- 
search on teaching effectiveness. In "How Teach- 
■ers Make a Difference." U.S. Government 
Printing Office, Washington, ' D.C, Stock No. 
1780-0813. 1971. 

Houston, W. R. and others. "Resources for |»e^;f6rm- ' 

ance-Based Educjition." The State Education "De- 
_partment. Division of Teacher Education and Cer- 
^^iification. Albany, NvY. 1973. 



RcsenKhlne, B. "Teaching Behaviors and Student 
Achievement." New York City: Humanities Press, 
1971. . 

Roseibhine, B. and FursI, N. Research on teacher per- 
formance criteria. In B. O, Smith (Ed.) "Re- 
search in Teacher Education: A Symposium." 
Englcwood Cliffs, N.J.; Prentice-Hall,. 1971. pp. 
37-72. 

Rosenshine, B. and Forsf, N. The use of direct obser- 
vation to study teaching. In R. M. W. Travers 
(Ed.) "Second Handbook of Research on Teach- 
ing." Chicago: Rand McNally and Co., 1973. 
pp. 122M84. 

Soboi« T. T.>"Student Handbook for General Teaching 
Skills Clinic." Miami: College of Education, Fior- 
. ida International University. 1972. ; 



ERIC 



34 



Alternative Teacher Assessment Strategies' 

By ' . - 

w. James popham 



At a tkne when the Nation.'^ entire educational en- 
terprise is being subjected to increasing public scrutiny, 
it is not surprising that the classroom teacher— gener- 
ally conceded to be the pivotal figure in most instruc- 
tional settings— is being evaluated more frequently and 
more rigorously than ever before. » Perhaps this 
increased stress on teacher evaluation stems frbni .the 
widespread concern about accountability in education. 
Possibly it is the predictable concomitant of |i glutted 
teacher market where employers can finally be choosy, 
when in former years their chief concern was to get an 
irtstructor, any instructor, to cover every classroom. 

Whatever the cause, concerns about teacher evalua- 
tion have become far more pronounced in the past few 
years than at any tiike during this century, even though 
educational researchers have been continually carrying 
out teacher effectiveness investigatiorfS for well over 
70 yeare. The difference in thS focus of these- p.ctivities 
provides the key to our understanding of why today's 
typical teacher starts to perspire a bit when someone' 
mentions teacher competence assessment. In the old 
days, most tpacher effectiveness researchers were 
searching for a suitable criterion variable which, if lo- 
cated. wouW permit them to isolate independent varia- 
bles (such as teachers' personality traits, educational 
experience, or instructional styles) that would contrib- 
ute to more effective instruction. Such investigations 
were accurately perceived by teachers as research in- 
quiries and, as such, were not viewed as particularly 
threatening. Even in' those instances where the aiten- 
tion of the investigator was clearly directed toward 
teacher evaluation; few teachers were very concerned. 
After all, even if defensible assessment techniques were 
discovered, it was generally held'that teacher evalua- 
tion efforts would be directed toward helping teachers, 
never firing them. The American public had great conr 
fidence in tlie Nation's public schools; and, although 



* An invited working paper for a meeting of the Multi-State 
Consortium on Performance-Based Teacher Evaluation, New 
Orleans, February 25-28, 1973. 



everyone knew there were differences in the abilities of 
teachers, a tenured teacher's position was next to in- 
violate. For example, a recent search ' of California's 
teacher employment records revealed that during the 
last 40 years Jiot one California teacher has been dis- 
missed on the grounds of incompetence. It is small 
wonder that in the past our teachers have not been too 
threatened by teacher evaluation activities. 

But they are threatened now — and with good rea- 
son. Dissatisfied legislators in a good many states are 
be^nning to ♦enact penalty-laden laws which call for 
more stringent teacher • accountability. The most cele- 
brated of these recent teacher accountability laws is the 
so-called Stull Act (naned after its author. Assembly- 
man John Stull) passec by the California legislature, 
during the 1971 legisl^:ive session (Assembly Bill 
293). The Stull Act has generated an immense amount 
of educational activity i among California school peo-^ 
pie,- for its implications are serious indeed. The new* 
laW calls for the annual evaluation of all probationary 
teachers and the biennial evaluation of all nonproba- 
tionary teachers. The evaluations must be made on the 
basis, as stipulated by law, of pupil progress according 
to district-explicated standards of achievement in all 
areas ,of. sturdy. What has happened in California as a 
result of the ituU Act is ihat an auempt has been 
made to opemiorudize incompetence so that even ten- 
ured teachers can be- dismissed if they are evaluated 
advereely. We can expect to see other state legislatures 
enacting comparable teacher evaluation laws in the 
next few years, particulariy if the California experiment 
seems to be working. 

But even if no more states establish teacher ap- 
praisal systems, there is still a strong likelihood that 
local ^districts, perhaps buffeted by school board pres- 
. sures, will set up some sort of teacher evaluation sys- 

» Personal communication, •Research Department, California 
Teachers Association, Burlingame, Calif. 

"2 See W. J. Popham, "California's New Precedent-Setting 
Teacher Mvaluation Law," Educational Research, Vol. 2, No. 
7, July 1972; pp. 13-15. 
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tern. In view of these developments at the state and 
loeal levels it does not require mueh preseienee to be 
able to foreeasl an inereasing need for the teehnieal de- 
viei-s and procedures required tor etleetive teacher ap- 
praisal systems* 

Although it is generally assumed by most laymen 
' (and many legislators) that educators currently possess 
adequate devices for use in evaluating a teacher s in- 
structional etfectiveness« nothing could be farther from 
the truth. The hFstory of teacher etTectiveness research 
is replete with failure after* failure in efforts to devise 
defensible indicators of how well a teacher teaches. 
Space limitations preclude an exhaustive analysis ^* of 
the limitations of previously tried assessment schemes, 
but each of the chief contenders, that is, ratings, obser- 
vations, and pupil test performance, have fatal defects. 

Ratings 

Briefly, the difficulty with ratings of teacher efTec- 
tiveness (characteristically supplied by administrators, 
but also obtainable from students, peers, etc.) is that 
different titers have different notions regarding what it 
is that constitutes good teaching. The same teacher 
who is rated high by one individual because of suw*h 
factors as "flexible interaction with learners** and "per- 
sonable, inforrfial rapport with class** may be rated low 
by another individual because of "poor discipline** and 
"classroom anarchy.*' We all use our private value ma- 
trix in judging whether good teaching has taken place, 
and when we try to pool these disparate sets of rater 
expectations, chaos is the characteristic result. How 
many times, fl^ example, has a classroom ^eacher been 
rated negativcly>by a principal because the leacher was 
conducting class in a way other than the manner in 
which the principal recalled his/her lustrous days in 
the ciassrcH>m. Yet that same teacher may receive a 
positive rating by the district office supervisor who has 
a different idea of how teaching should be carried on. 
It has been observed that one person*s humorist is an- 
other person's smart aleck. Similarly, one rater*s Mr. 
Chips is another rater*s Mr. Peepers. 

Observations 

With respect to systematic observations of the .teach- 
er*s classroom behavior, we encounter an interesting 

* For a more detailed examination, of the strengths and 
weaknesses of various teacher effectiveness assessment ap- 
proaches, sze J. D. McNeil and W. J. Popham, 'The Assess- 
ment of Teacher Competence,** Chapter 7, second edition. The 
Handbook of Research on leaching, R. M. W. Travers ed., 
Nfacmtllan, 1973. 
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assumption. It runs as follows: if certain process varia- 
bles can be found to correlate positively with desired 
outcome variables, then by ascertaining whether those 
process variables arc present, on that basis alone we 
can make judgments regarding the desired outcomes. 
For example, if it is discovered that a teacher's provi- 
sion of practice opportunities for learners generally re- 
sults in desirable learner attainment, then proponents 
of observational teacher evaluation schemes would cfOn- 
tend that we can, at least in part, evaluate teachers on 
the basis of the degree to which they provide practice 
opportunities. 

The trouble with the logic of this approach is its 
tendency fb force one to the position that the process 
variables scrutinized by classroom observers are not 
only necessary for securing worthwhile results with 
learners, but that they are essentially sufficient* For if 
the phenomena observed; e.g., amount of teacher talk» 
are viewed as means to an end, why not as$ess the end 
results directly without encountering the measurement 
noise associated with the extra assessment step* For al- 
though no upstandihg classroom observation devotee 
will ever assert that those behaviors observed are with- 
out exception associated with desired outcomes in 
learners (such as important cognitive or affective 
changes), the logic of the obs'ervation strategy pushes 
us to place greater reliance on means-end predictive re- 
lationships than the current sophistication of our obser- 
vational techniques permits. If we are really interested 
in the ends, why not focus our assessment energy on 
them? 

A second difficulty with observatioo-based ap- 
proaches to teacher appraisal is that although a teacher 
may display optimal use of the classroom behaviors 
called for in the observation system, there may be dele- 
terious factors present, factors not built into the obser- 
vation structure, whose presence will essentially cancel 
out the positive features of the teacher's classroom be- 
havior. The only way to head off this assessment diffi- 
culty would be to build an observation system so ex- 
haustive that it could pick up all (or most) negative 
process variables, but by that time the system would be 
too vast to be practical. 

Another difficulty with observational approaches to 
the assessment of teacher effectiveness is that whereas 
they might prove useful in identifying some cliissroom 
practices which in general will yield beneficial results 
with learners, the teacher evaluation game demands 
personal and particular decisions, not general guide- 
lines. A particular teacher working toward particular 
goals with particular students in a particular setting 
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may break ait the process guidelines and yet achieve 
superb results. The particularized interaction effects are 
too subtle for ouf currently unsophisticated observation 
systems. There have been several outstanding pro foot- 
ball quarterbacks whose passing form looked abysmal, 
yet when tht receiver arrived at the appointed spot the 
ball was always there waiting. 

Finally, there is considerable danger that when the 
stakes are high enough (and job security represen:s a 
big bet), many teachers will **fake good.** Observation 
evaluation systems are particularly susceptible to such 
faking, for in these days of openiy described criteria 
we can expect teachers to know what factors will be 
involved in the observation system. Indeed, any dili- 
gent and legally informed teachers organization should 
be easily able to uneartn the observation dimensions 
involved. Having been apprised of v *iat p actices yield 
positive eva!uati\>ns, is it so. unrealistic tc expect that 
teachers will tend toward the use of those practices 
when under observation? Of course, if one wished to 
employ constant monitortug of classroom behavior 
through such devices^^as closed circuit television, then 
such fakcability fears would be vitiated, but by then 
most schools would have been closed permanently be- 
cause of the ami- 1984 teachers* strikes. 

Pilpil Test Perfomiance 

The chief deficiency with the use of student test per- 
formance as an index of teacher proficiency has gener- 
ally been that the wrong kinds of tests were employed. 
Since 1900 most teacher effectiveness research in 
which pupil test performance was employed as a crite- 
rion variable involved the use of standardized achieve- 
ment tests. Since most standardized tests were designed 
to serve a different purpose; namely, to permit compar- 
isons among individual learners (not among teachers) 
they invariably resulted in a **no significant different*' 
outcome. 

The difficulties with standardized or norm-referenced 
tests, particularly for teacher evaluation, have been 
treated elsewhere/ but their two most visible defects 
can be briefly identified. First, since commercially de-' 
velopcd standardized tests must serve students through- 
out an entire nation, the generalized naiure of their 
content coverage Ls often inconsistent with local curric- 

^See, for example, W. J. Popham, "Domain-Referenced 
Measurement and Teacher Evaluation^ Education Technol- 
ogy, in press: Robert Glascr, "A Criterion-Referenced Test," 
Crtterion*Refeix^nced Measurement: An Introduction. J. 
Popham, cd.. Educational Technology Publications, Englewood 
Cliffs, N.J., 1971, pp. 41-51. 



ular emphases. Incongruent measurement and curricu- 
lum results in misleadii^ data. Second, certain psy- 
chometric properties of norm-referenced tests (such as 
their heavy reliance on producing among-learner vari- 
ance) leads to tests which are sometimes insensitive to 
detecting the results of high quality instruction. 

In the past few years the development of criterion- 
referenced (or mastery) tests offers teacher e valuators 
an alternative to standardized tests for assessing an in* 
structor's impact on learners. The judicious employ- 
ment of d"iterion-referenced tests for teacher evaluation 
purposes is only beginning to be seriously investigated. 

Teaching Performance Tests 

In the mid-sixties, the writer had reached a point of 
frustration regarding teacher effectiveness assessment 
devices and, after a reappraisal of alternative assess- 
ment strategies, had proposed the development of an 
alternative approach to solving this problem; namely, 
throu^ the use of a teaching performance test. Two 
separate projects ^ were supported by the U.S. Office 
of Education, each designed, to develop and attempt to 
validate teaching performance tests hIv different subject 
matter fields. While the rationale underlying the teach- 
ing performance .test strategy, as well as the detailed 
results of these two projects are supplied elsewhere,^ a 
^ef description of the performance test approach can 
^'^pplied here. * 

One of the major difldculties in c<Mnparing teachers 
for purposes of instructor evaluation is that diiferent 
teachers have different instructional emphases, thereby 
making across-the-board comparisons misleading. The 
teaching performance test counteracts this problem by 
providing an identical task for dilS^rent instructor?; 
namely, the ability to accomplish prespecified instruc«- 
tional objectives. The teaching performance test is built 
on the general premise that one chief reason for a 
teacher's existence in the classroom is to bring about 
worthwhile changes in students; that is, changes in 
their knowledge, attitudes, skills, etc. To the extent 
that this is true, then one criterion by which a teacher 
should be judged is his or her ability to bring about 
such changes. By providing identical instructional 

Performance Tests of Instructor Competence for Trade or 
Technical Education, USOE Cooperative Research Contract 
No. OK-5-85-051; Development of a Performance Test of 
Teaching Proficiency, USOE Cooperative Research Contract 
No. 3200. 

"W. J. Popham, "Performance Tests* of Teaching Profit 
cicncy: Kattonate. Development, and Validation.'* American 
Educational Research Journal, January 1971, 8 (1). pp. 105- 
117. 
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objcctivcj; for'tcachcrs, then giving the teachers an op- 
portunity to accomplish thone objectives using whatever 
instructional techniques they wish, a measure of the 
teacher s ability to accomplish given objectives can be 
provided. One might wish to argue that the better 
achiever of given objectives will also be the better 
achiever of his/her own objectives, but this is a ques- 
tion which can be answered empirically. If one simply 
decides that an important criterion of teaching is the 
ability to accomplish instructional objectives, then 
teaching performance tests would appear to have some 
utility in a data-based evaluation matrix. 

The steps involved in a teaching performance test 
are these: (1) the teacher is provided with an explicit 
instructional objective (and sample test item) along 
with any background information necessary to become 
familiar with the subject matter related to that objec- 
tive; (2) the teacher plans a lesson designed to accom- 
plish the objective: (3) the teacher instructs a group of 
learners, typically a small group of learners for a short 
period of time; (4) the learners are posttested with an 
examination based on the objective. The examination 
has not previously been seen by the teacher but its na- 
ture is readily inferable from the objective (and sample 
test item) previously given to the teacher. 

In the USOE-supported research studies described 
above, the purpose of developing the performance tests 
was primarily research*oriented; that is, it was antici- 
pated that these devices would be employed principally 
for research purposes such as the identification of rele- 
vant independent variables. Consistent with that intent, 
the performance tests involved in those investigations 
consumed a fairly large amount of learner instructional 
time, ranging from 4 to 10 hours. At the conclusion of 
those investigations, it became, clear that if teaching 
performance tests were to prove practical for teacher 
evaluation or instructional improvement purposes, they 
would have to be developed for much shorter periods 
of irstructionai time. As a consequence, the wrlter^s re- 
cent development work with performance tests has fea- 
tured instruments which take only 15 minutes of in- 
struct onal time and are designed to be used with small 
groups or adults or younfger learners. These teaching 
performance tests, frequently referred to as instruc- 
tional mini-lessons, superficially appear comparable to 
the microtefaching exercises developed at Stanford Uni- 
versity some years back, fn rationale, however, they 
are quite different. The Stanford microteaching lessons 
emphasize the teacher^s acquisition of process skills; 
e.g., good questioning techniques. The instructional 
mini-lessons a*f erred to heie, on ♦h^ other hand, focus 



more heavily on the results of the teaching than upon 
the instructional procedures themselves. 

During the past few years teaching performance tests 
have been employed both in preservice and inservicc 
teacher education settings/ Get^rally speaking, these 
performance tests have been of the short duration al- 
luded to abov&; i.e., IS to 20 minOtes in length. But 
while thesc^evices appear useful in instructional set- 
tings, for example, in helping prospective teachers be- 
come more facile at accomplishing prespecified instruc- 
tional objectives, their utility for purposes of teacher 
evaluation has been largely unstudied. 

In a recent paper ^ Glass has proffered the notion 
that teaching performance tests may have insufficient 
reliability to permit their effective use in teacher evalu- 
ation enterprises. Glass cited several investigations in 
which the reliability of teaching performance tests was 
clearly inadequate. Several of the investigations cited, 
however, had been conducted as doctoral dissertations 
or by novice researchers. The reliability of teaching 
performance tests is as yet a seriously unstudied mat-* 
ter. For one thing, the teaching performance tests used 
In these investigations have been constructed on an al- 
most opportunistic basis, that is, whatever topics, 
objectives, etc., have come to the investigator's mind. 
No attempt has been made to carefully delineate t||e 
truly critical dimensions in teaching performance tests* 
Beyond that, only one investigator^ has carefully at- 
tempted to study the reusability of even these' ill-define<i 
performance tests. Results of tftis investigation will be 
reported by Millman at the 1973 meeting of the Amer- 
ican Educational Research Association. Examination of 
the Millman findings suggests that the reliability evi- 
dence, once again, is not encouraging. But, as indicated 
above, the nature of the performance test employeld in 
that invesiigation was not rigorously explicated. 

When this paper was solicited as one of several deal- 
ing with the ''state of the art'' in the assessment strate- 
gies suitable for performance-based teacher education, 
I had just completed the final draft of an American 

^W. J. Popham, Applications of Teaching Performance 
Tests to inservice and Preservice Teacher Education. A paper 
presented at the annual meettag of the American Educational 
Research Association. New Orleans. February 26-Mnrch I, 
1973. 

""Ciene V. Glass. Statistical and Measurement Problems in 
Implementing the Stull Act, Stanford University Invitational 
Conference on the StuH Act, October 1972. Palo Alto, Calif. 

-'Jason Millman, Psychometric Characteristics of Perform- 
ance Tests of Teaching Effectiveness. A paper presented at the 
annual meeting of the American Educational Research Asso- 
ciatlon« New Orleans^ February 1973. 
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Educational Research Association (AERA) paper de* 
scribing a set of minimal competencies for a perform- 
ance-based teacher education program.'". I had even 
sketched alternative assessment procedures for each of 
the competencies. Now I just couldn't bring myself to 



^"W. I. Popham, Identification and Assessment of Minimal 
Competencies for Objectives-Oriented Teacher Education Pro* 
grams. A paper presented at the annual meeting of the Ameri- 
can Educational Research Association, New Orleans, February 
1973. 



rewrite the paper or even to subtly paraphrase my orig- 
inal paper. I try to restrict my paraphrasing talents to 
the writing of others, not my own. " • 

Accordingly, in the present effort I have attempted 
to focus exclusively on the major assessment alternatives 
for teacher competence appraisal. Since if perform- 
ance-based teacher education programs cannot demon- 
strate that their competency-armed products are indeed 
better teachers, then the performance-based teacher ed- 
ucation folk had best fold up their competencies and 
slip away into the night. 
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Applications Of Teaching Performance Tests To 
Inservice And Preservice Teacher Education* 

By 

W. JAMES POPHAM 



Irrespective of whether the programs are referred to 
as **performance-based," **coinpetency-based," or 
**skin-foci»scd/* the advocacy of teacher preparation 
schemes designed to promote measurable capabilities 
of instructors^ is beginning to be quite fashionable' 
among teacher educators. A recent survey ^ of Califor* 
Dia teacher education institutions found that two*thirds 
of the institutions participating in the survey indicated 
they were engaged at least partially in ccnnpetency* 
based teacher preparation; There is even a multi-state 
consortium - on performance-based teacher education 
featuring such forward-looking states as' Arizona, Flor- 
ida, Minnesota, New York, Oregon, Texas, Utah, Ver- 
mont, and Washington. Surely, if most teacher educa- 
tors are not caught, up in the competency-based game, 
then we are at least witnessing the pregame warmups. 

Whether competency-based teacher education will 
become something other th^ one of those ephemeral 
fads, so common in teacher education remains ^ to be 
seeq. Although many teacher educa ^rs are quick to 
join the competency-based movement at the verbal 
level, few are willing to devote the requisite energy to 
devising the criterion measures without which the ap- 
proach is only rhetoric. In the recent survey ^ of Cali- 
fornia teacher education colleges and universities, no 
institution reported satisfactory assessment procedures 
for a competency-based program. As with so many ed- 
ucational innovations, advocacy is less expensive than 
workable implementation procedures. 



* A paper presented at the annual meeting of the American 
Educational Research Association^ New Orleans, February 26- 
March 1, 1973. 

^ Performance*Based Teacher Education, Vol. 1, No. S» De- 
cember I972» pp. 1,5. 

^ Multi-State Consortium on Performance-Based Teacher 
Education, c/o Bureau of Teacher Education, New York State 
Education Department, Albany, N.Y. 

» Op. cit. 



Teacher Competencies as Enabling Skills 

But there is another aspect of the competency-based 
teacher education movement which is equally intrigu-* 
ing. If you sit in on almost any discussion among pro- 
ponents of competency«-based teacher education, you 
will hear them describing the teacher competencies 
they are trying to promote as though they were ends in 
themselves. As with many recent reli^ous converts^ the 
fervor of these teacher educators for competency iden-> 
tihcation has become so ah-consuming that they losQ 
sight of what the competencies are really supposed to 
accomplish. 

Reducing the problem to its essentials, we can see 
; that whatever competencies a teacher acquires must be 
viewed as vehicles for making that teacher more effec* 
tlve. More effective, in this instance* means a teacher 
better able to help learners. Thus, the competencies 
most frequently identified by performance-prone 
teacher educators are really^ en route skills which 
should contribute to the Jterminal skill of being able to 
help learners. For example, suj^pose one of the teacher 
competencies we are trying to promote invc^ves the 
teacher*s ability to view real or simulated instructional 
situations and identify the extent to which certain in- 
structional tactics have been employed. We assume 
that the teacher who can master such a skill will subse- 
quently be able to apply this skill in real instructional 
»tuadons. Such skills should thus be viewed as precUir- 
sive to one*s becoming an effecth^e teacher. 

Now the point of this distinction between en route 
and terminal competencies is that the bulk of cottipe- 
tencies currently viewed as the staples of performance- 
based teacher education are well removed from those 
which might legitimately be viewed as terminal. Unless 
en route skills are constantly verified asjbeing actual 
contributors to terminal skills, then we have little as- 
surance that defensible competencies are being pro- 
moted* 



40 



Some of the competencies sought by teacher educa- 
tors should be closer to the terminal proficiencies we 
wish teachers to display. The purpose of this paper is 
to explore the teacher education applications of one 
measurement approach designed to assess such a near- 
terminal competency. The measurement approach 
under consideration is the teaching performance test. 

Teaching Perfformance Tests: 
Description and Rationale 

In brief/ teaching performance tests work as fol- 
lows: An instructor is presented with one or more ex- 
plicit instructional objectives ^plus a sample test item) 
and is directed to prepare a short lesson designed to 
accomplish the objective (s). If the obj^ctiveVdeals with 
a topic presumed to be unfamiliar to the teacher, then 
relevant background information is made available. 
After planning the lesson, the teacher instructs a group 
of learners (either children or adults) for a short pe- 
riod of time: e.g., 15 minutes. The number of learners 
can be as few as a half dozen or as many as an entire 
class. At the conclusion of thejesson the learners are 
given a posttest based on the objective. While not 
previously seen by the teacher, the nature of the test is 
readily inferable from the objective the teacher has 
been attempting to achieve. Characteristically, learners 
are also asked to rate how interesting the lesson was. If 
such an interest rating is employed, the teacher is ap- 
prised of the forthcoming rating and«encouraged to de- 
sign a lesson which not only accomplishes the cognitive 
objective, but also promotes positive learner interest 
ratings. 

Using these two indicators, an estimate is, provided 
of the teacher's ability to promote prespe^ified objec- 
tives, both cognitive (as reflected by posttest perform- 
ance) and affective (as reflected by the interest rat- 
ings). Now it may be argued that an unrepresentative 
estimate of instructional prowess is yielded by measur- 
ing a teacher's ability to promote learner attainment of 
prespecifled objectives for a small group of learners 
during a brief time period. Yet, while perhaps not as 
representative of the real teaching worid as we might 
wish, there are sufficient parallels with reality >hat 
^uch an assessment procedure may have utility lor 
teacher educators. In particular, since it more closely 

*T. EiTMcNcil, and J. Popham, *The Assessment of 
Teacher Contpetence/* Chapter 7,. The Handbook of Research 
on Teaching. R. M. V^. Travcrs, cd.» Mactniilan, in press. * 

» If novel subject matter is employed, no pretext is typically 
employed. With less esoteric topics, a pretest may be utilized 
to identify suflteiently naive learners. 



approximates a terminal teaching skill than many of 
the competencies currently being fostered by perform- 
ance-based teacher educators, it may have advanta- 
geous instructional and cvaluational dividends. The re- 
mainder of this analysis will (1) set forth three 
distinctive applications of teaching performance tests in 
teacher education operations, (2) describe actual inser- 
vice and preservice situations in which performance 
tests have been employed, (3) identify usage guidelines 
derived from these experiences, and (4) discuss certain 
problems which have arisen in the use of teaching per- 
formance tests. 

Application One: 

A Focusing Mechanism 

It is the writer^s belief that much of the ed|icati6nal 
ineffectiveness which exists in our schools c$n be at- 
tribute<! directly to teachers' preoccupations with in- 
structional process. Far too many teachew are caught 
up with concerns about devising new and exciting ways 
of teaching, without ever verifying what effects those 
procedures have on children. Some teachers pridefuliy 
announce that they strive to •Heach their class differ-^ 
ently every year,*' never recognizing that they may be 
abandoning one year an approach that was truly effec- 
tive the previous year. Innovations are adulated for 
ttvcir own sake. For instance, open schools are cur- 
rently in vogue. Ten years ago, it was nongraded 
schools! A decade eariier, we were praising the rap- 
tures of the core curriculum. 

Not that there is anything intrinsically wrong witi; 
these new instructional approaches, for surety they 
possess many meritorious features. Jt's just that too 
many educators succumb to the lure of an attractive 
instructional proc(^ without checking the quality of its 
impact on learners.' And that, after aii, should be the 
reason we search for bettef instructional procedures. 

Heoce, as a method of counteracting what appears 
to be an almost hereditary concern about instructional 
process, frequent use of teaching performance tests can 
provide a mechanise to focus the teacher's attention 
on the effects of instruction. Since in a performance 
test situation, the quality of a teacher's efforts is predi- 
cated on results achieved with learners, both cognitive 
and affective, it is difficult to discount the effects of in- 
struction on pupils. For example^ if you are a prospec- 
tive teacher who during a semester is obliged to teach 
a half dozen or more mini-lessons (as short duration 
teaching performance tests are sometimes called), and 
the first concern after your lesson is a jdetermination of 
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its effects on learners, it is dilticult to see how you 
would not soon begin to view as important a lesson's 
impact on pupils. 

This initial application, therefore, is instructional in 
nature. More specifically, it is designed to foster a dis- 
position on the part of the teacher; namely, a disposi- 
tion to view as important the effects of instruction on 
learners. 

Application Two: 

A Setting for Testing the Value of 

Instructional Tactics 

liven though the foregoing application; i.e.« as a 
mechanism for focusing teachers* attention on the con- 
sequences of instruction, may have suggcsti5d that at- 
tention to instructional procedures was somehow repre- 
hensible, such is surely not the c^ase. We can only 
secure good results with learners if we use appropriate 
instructional processes. The trick is to apply instruc- 
tional techniques judiciously in such a way that we can 
either verify their efficacy (in terms of effects on learn- 
ers) or at least be able to make high probability 
guesses that a given technique will yield desirable re- 
sults with pupils. 

A second application of teaching performance tests 
involves their use as a method of allowing teachers to 
test the differential effectiveness of vbHous instructional 
techniques. Teachers can compiute a series of mini-les- 
sons attempting to incorporate different instructional 
tactics, then judge their worth in terms of the results 
yielded with learners. For instance, suppose a teacher 
taught the same mini-lessons to two different groups of 
comparable learners, the lesson being essentially the 
same except that one lesson provided much opportu- 
nity for learners to practice the skills called for in the 
mini-ipsson's objective, while the other lesson provided 
no such practice. The teacher could then contrast the 
posttest results of both lessons and begin to reach a 
conclusion regarding, for certain kinds of instructional 
objectives, the efficacy of providing relevant practice. 

Not that a tactic-present versus tactic-absent design 
must be employed in this second, application of per- 
formance tests, for a teacher can often gain insights re* 
garding the value of a given instructional procedure 
from using the procedure even without the control 
treatment. This is particularly true when for a particu- 
kir performance test there are some normative data 
which, even in rough terms, yield an estimate of how 
well teachers typically perform on the lesson. This 
point will be treated in more detail later. 



An important aspect of this application of teaching 
performance tests is that a teacher need not be the ac«- 
tual instructor in a mini-lesson to profit from the 
mini-lcsson teacher's experience. A typical format for 
the conduct of mini-lessons, either those taught to 
younger learners or to a grour of one's peers, is the 
postlesson analysis session. Dunng this session, the 
teacher's instructional approach, is appraised in terms 
of its effects on learners. Many teachers report they 
learn as much from watching the mini-lesson teacher's 
lesson, then analyzing its strengths or shortcomings* as 
they do from actually teaching the lesson themselves. 

Frequent teacher performance tests, either for teach- 
ers as mini-lesson instructors or as an object for group 
analysis of other's instructional efforts, can provide the 
focus of a consequence-oriented preservice or inservice 
teachers education program. 

Application Three: ^ 
A Formative or Summative Program 
Evaluation Device , 

.V ^ ^ 
Developmental work with teaching performance tests 

is still at such an early state that it may be imprudent 
to employ them for the evaltiation of individual -teach-* 
ers. The only exception might be for isolating instruc- 
tors who are extremely weak of strong in their ability 
to accomplish prespecified goals. Nonetheless, as a pro- 
gram evaluation assessment technique, ^performance 
tests may have considerable utility. Indeed, the most 
important use of teaching performance tests may be as 
instruments to aid in the appraisal of inservice or pre- 
service teacher education enterprises. 

The argument, briefly, is that if a teacher education 
program sets out to promote teachers' abiiit«es to ac^ 
compljsh prespecified objectives, then the pnigram can 
be legitimately evaluated in terms of its abiliiy to do 
so. Here's how the evaluation strategy might work. At 
the outset of a teacher education program; e.g., a pre- 
service credential sequence or an extended staff devel- 
opment institute, a representative sample (or all) oi 
the participating teachers could complete one of two 
different performance tests (e.g., test X and test Z, 
with half the teachers completing test X and half test 
Z). At the close of the program, the teachers would 
complete the other performance test. The prediction 
would be that for both tests the teachers' post-program 
efforts would produce markedly better results than the 
pre-program .efforts. As a summative evaluation strata* 
gem. such an approach could yield devastating infor- 
mation. What^happens, for instance, if a teacMbr educa- 
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tion program discovers its teachers are no better able 
after intensive Instruction to promote learner mastery 
of objectives than they were prior to instruction? 
Surely drastic changes in the program Seem warranted. 
Perhaps* if such failures are recurring phenomena, the 
program should be terminated. 

If performance tests are employed at earlier stages 
of the program, with a view to guiding program modi- 
fications during i\c course of the instruction, then 
formative evaluat on benefits can also be derived from 
the use of teaching performance tests. 

In review, then, we have briefly examined three pos- 
sible applications, of teaching performance tests in 
connection with either preservice or inservice teacher 
education efforts. There wUl, of course, be other uses 
of performance tests in teacher education. For exam- 
pie, a professor might use teaching ped[ormance tests 
as a motivating mechanism, showing ^vice students 
that more skilled teachers can outperform beginners in 
su^ instructional situations. But focusing for the mo- 
ment on the three application strategies described thus 
far, we can examine some actual utilizations of teach- 
ing pe^ormance tests. 

Actual Inservice and Preservice Applications 

The J0fmn Complex. One of the first reported uses - 
^f teaching performance tests to aid practicing teachers 
tbdk place during 197*1 in the Jordan Complex, an af- 
filiated group of urban schools in the Los Angeles City 
School. District. Under 'the leadership of La Verne 
Parks, complex director, groups of teachers met weekly 
after regular school hours to witness each other taking 
turns teaching miii-lessons, then discussing the merits 
of the teaching approaches. Lessons were divided 
about evenly between those designed for young chil- 
dren (volunteer pupils were used as learners) and 
those designed for adults (teachers participating in the 
program took turns serving as learners). 

Reactions tp the program, extending over several 
months, were quite positive. Merle Williamson and 
Joyce Cooper, the class leaders, gathered anonymously 
supplied course evaluation data indicating that 84 per- 
cent of the participating teachers feIC the class had 
helped them (1) understand the role of instructional 
objectives, (2) plan lessons for given periods of time, 

(3) develop alternative instructionaS strategies, and 

(4) critique lessons on the basis of posttest results. 
Eighty-seven percent of the participants reported that 
the class had helped them personalize instruction. Mrs. 
Parks remarked, in reviewing the experience, that 



**One of the most exciting outcomes, from this course 
was that teachers began to look more critically at 
themselves and their peers in terms of factors contrib- 
uting to successful pupil results.** •* 

UCLA. As might be anticipated because of the writ- 
er's afiiliation, use of teaching performance rests in the 
teacher education program at UCLA over the past 
years has been fairly extensive. All three of the appli- 
cations discussed in the pr^ious section have been em- 
ployed, with attention generally given to the use of 
performance tests as an instructional intervention. 

Typically, teaching performance tests have ban 
used with preservice candidates, usually involving i^- 
sons taught to other members of the class. The setting 
for these lessons is ordinarily referred to as a mini-les- 
son clinic and features the customary teaching-testing- 
analysis model. Usually one or two mini-lessons are 
taught during a 2-hour clinic session. During some 
terms we have tried to squeeze three mini-lesson as- 
signments into a single 2-hour lab period. On other oc- 
cassions, we have required the prospective teachers to 
generate their own objectives and tests, then try to ac- 
complish the objectives in a short-term lesson, as an 
additional exercise in* prcnnoting learner goal attain- 
ment. 

The most recent procedure we have employed for 
Using teaching performance tests is described in some 
detail in the appendix. Briefly,. it involves the iise of 
weekly nine-student mini-lesson clinics * during which 
one preservice credential candidate teaches six class- 
mates while two classmates plus a teaching assistant 
serve as instructional analysts. In addition, each cre- 
dential candidate is obliged to teach at least three 
mini-lessons outside of class time to small groups of 
adults. Thus, in a 10-week academic quarter, prospec- 
tive teachers have about two dozen opportunities to 
serve as mini-lesson teachers, analysts, or learners. 

As our use of mini-lesson clinics has increased, there 
has naturally been great interest in the manner in 
which the teacher education students were receiving 
them. At the end of the fall quarter, a quarter during 
wHich mini-lessons were employed according to the 
scheme presented in the appendix, students were asked 
to supply anonymous evaluations of the course at its 
conclusion. Little structure in the evaluation form was 
presented to the students, only requests to isolate parts 
of the course they liked most, liked least, etc. Of the 
58 students who mentioned the mini-lesson clinics, 32 
were positive and 26 were negative. In view of the fact 

^ Personal communication to the writer. May 15, }972. 
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that none of the mini-lesson cHnie leaders possessed 
any prior experience with mini-Icsson clinics, either as 
participantH or supervisors, hence were probably less 
skillful as clinic leaders than might have been wished, 
these results are not distressing. 

One of the hopes in setting up mini-lesson assign- 
ments so that students were obliged to teach the same 
mini-lesson a second time (outside of class time) was 
that the mini-lessoa, teacher would profit from the 
clinic critique session. Hopefully, the insights gained 
from the analysis of the teacher's lesson would lead to 
instructional improvements when the mjni-lesson ^as 
retaught and a comparable posttest form was iui^d'to 
assess learner achievement. 



T^tel. Results of Mini-lessons Retaiu^t 
After Initial Mini-lesson Clinic Analys^v 





First 




Second 




Lesson 


No 


Lesson 




Superior 


Difference 


Superior 


Cognitive Posttest 
Interest Rating 


•> 

n»18 


n«6' 


n«31 


n»t3 




n«9 


n«33 



Happily, this appears to have been the case in the 
fall 1972 UCLA situation for, from the reports sup- 
plied by class members (to which no grade credit was 
applied), both cognitive and affective performances in- 
creased more frequently pn the retaught Messons than 
they decreased, as can be seen in table l! Of course, 
these data are only- suggestive in view of the fact that 
there were no controls exercised over the ability of 
learners in the retaught or initially taught lessons. 

Recently we have also used performance tests in 
connection with program evaluation. During the fall 
1972 quarter, six mini-lessons were randomly assigned 
to approximately a dozen credential candidates (from 
a particular teacher education course) at the beginning 
'of the quarter and to a different group of credential 
candidates at or near the close of the quarter. In gross 
terms, the prediction was that the performance of these 
prospective teachers would be better later in the quar- 
ter, presumably aft?r the impact of the teacher educa- 
tion program had worked its ''beneficiaP* effects. 

Since the mini-lessons were to be taught to class- 
mates, care had to be taken not to involve any students 
in mini-iessons at the first of the quarter (either 
teachers or learners) which they would encounter at 
the close of the quarter. Sirjpe six different mini-lessons 
were u.scd, this posed no problem. Although it had 
been planned to havejhree teachers per mint-lesson on 
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both an early*inH:ourse and late«ln-course basb, with 
student attrition, missed assignments, etc., a less bal- 
anced performance .distributibn resulted. The minifies- 
^ sons were taught to groups of approximately six learn* 
ers (according to the procedural scheme described in 
the appendix), mth the early«in-oourse mini^lessons 
occurring during the second week on the quarter. Ide- 
ally, the first week would have been used for the pre- 
test mini-*Iesson, but course organizational requirements 
dictated that the seccmd week was a more reasonable 
choice. Mini-lessons completed during weeks eight, 
nine, or 10 of the 10-week quarter ccmstituted 4he 
late-in-course measures. Since all mini-lessons were 
completed on Fridays, this means that prior to the late 
course mini-lessons the credential candidates had typi- 
cally experienced a minimum of about 8 weeks of in- 
struction. The teacher education program variation 
under consideration involved daily 2-hour class ses- 
sions, including lectures, discussions, observations in 
public schools, and the mini-lesson clinic activities. 

Results of the early course versus late course meas- 
urement ai^c presented in table 2 for both the cognitive 
measure (posttest percentage correct) and affective 
measure (interest rating). All data included in table 2 
are mean results for a given mini-iesson teacher, typi- 
cally based on an n of five or six classmate-learners: It 
should be noted that since six different mini-lessons 
were employed, with different levels of difficulty and 
interest associated with each, interpretation of the data 
should focus on the columns qf the table. 
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Table 2. Mean Results of Early-iit^Course and Late4n-Course 
MioMessons Tauftht by Credential Candidates in a 
UCLA Teacher Education Pnwam 



Cognitive Results 


Xtini-li'sstms 


1 


2 


3 


4 


5 


6 


Pastiest Percentages Correct 


Early*in*Course 
Late-in-Course 


72 
74 
80 


90 


36 


84 
90 
96 


82 


60 
74 


88 
90 
92 


73 
95 


36. 
SO 
65 
67 


86 
97 


82 


82 


\ ' 

\ 

\ti^ective Results 


Minhlessons 


1 


... J 

2 


3 


4 


5 


6 


Ratings of Lesson's interest 
(5 = hi, ! = low) 


Eariy-in-Coikse 

\ 

\ 

Lwte-tn-Course ^ 


2.7 
3.2 
3.4 


3.8 


3.8 


3.6 
3.8 
4.0 


3.0 


•2.i 
3.8 


3.0 
3.8 
4 0 


3.3 


3.9 
5.0 
4.1 


4.0 


3.2 


4.6 

1 



An examination of tabic 2 will reveal that the pre- 
dicted results were supjported by the data, Fdr both in- 
dices; i.e., mean posttest percentages correct and inter- 
est ratings, the late-in-course performances exceeded 
the early-ifi-course performances. 

By converting the results for each performance test 
to standard scores for that measure^ with a mean of 50 



and a standard deviation of 10, it was then possible to 
pool the data from ail six performance tests and com- 
pute separate / tests for both the cognitive and affective 
rcsuU>. A summary of these analyses is presented in 
table 3 where it can be seen that a significant differ- 
ence was present in both instances, with^a mean differ* 
ence of .83 standard deviation unit present in 4he case 
of the cognitive measure ^nd .T>7 standard deviation 
unit for the affective measure. Both differences, as 
hoped, favored the late-in-course teaching performance 
tests. 

This represents the initial time,^ at least to the writ- 
er's knowledge* that teaching performance tests have 
been used in this, manner as a teacher education pro- 
gram evaluation technique. It is obvious, even from ex- 
anf|iniTig the data table, that scmie refinements are in 
order; e.g., the disparate w's in the columns, etc. Never- 
theless, as an illustration of the use of performance 
tests for program evaluation purposes, the foregoing 
description may be of some utility. 

California State University, Northridge. During sum- 
.me.r 1972, two inservice worksljops for approximately 
50 teachers and ^(^ninistrdtors were sponsored by Cal- 
ifornia State University, Northridge in coordination 
with Administrative Area K of the Los Angeles City 
Schools^ Clare Rose, the instructor for both workshops, 
reports that teaching performance tests were used in 
the workshq[)s as a vehicle for achieving one of the 
workshop goals; namely, that participants would be 
able to supervise objectives-based instructional im- 
provement programs. 

A demonstration mini-lesson was presented in each 
workshop, followed one week later by having all work- 
shop participants serve either, as mini-lesson teachers 
or students, as particular mint-lessons were taught 
twice by six volunteers. Prior to the second teaching of 



Table 3. Analysis of Early4n-Course and Late-in^Course MinMesson 
Results, Pooled on the Basis of Standard Score Transformations 



n 


X 


S.D. 


n 


45.46 


7.86 




53.77 


9.28 


11 


44.73 


5.59 


13 


54.46 


10.90 



P* 



Cognitive Results 
Early-in-Course 
Late-in-Course 

Affective Results 
Early-in-Coprse 
Late-in-Course 



8.31 
9.73 



2.25 
2.57 



<.025 
<.01 



* One-tailed. 



] each mini-lesson, an analysis of the initial lesson was 
carried out^ Professor Rose reports that 90 percent of 
the mtni-*le.sson teachers were able to promote higher 
posttest results on the second lesson. She indicated that 
100 percent of the workshop participants reported, on 
anonymous end-of-session evaluation forms, that the 
ttyni-Iesson activities had been valuable. 
• Professor Martin Levine has also employed teaching 
performance test* in preservice teacher education for 
several years at California State University, Northridge. 
Most recently, Professor Levine reported the following 
format for his use of'perforftianqe tests: 

l^y preservice- secondary education methn 
ods course meets qn the campus of a par- 
ticipating junior high school, usually for 
three hours weekly (e.g., Wednesday 9* 
12). One or two classes of secondary pu» 
pits are assigned to the college methods 
class by the principal.* Usually pupils in ^ 
these classes lare" classified as "'low ability 
learners'', in need of more individual atteh* 
tion. Pupils , are assigned at random to col* 
• lege trainees who are responsible for 
achieving, prespecified instructional objec* 
ttves issued by the*^ instructor. Each trainee 
^teaches his miniclass of three pupils for 
^ one Kpur during each weekly meeting of 
the course. Objectives deal -with general 
study skills and thus are appropriate for 
both pupils who need to master this kind- 
of objective and for college trainees who 
come from all of the different academic 
areas commonly: ^taught in secondary 
schools. Trainees have a week to prepare 
to teach each objective. A teaching time 
limit of thirty minutes or less is usually 
set. The instructor admititsters a pretest 
and posttest. Trainees analyze their teach- 
ing effectiveness in terms of how well their . 
pupils achieve the objectives. Trainees 
may re-teach objectives during s^t^equent 
nieetings in <iases which warrant it. From 
time to time, trainees combine miniclasses» 
with one member teaching to a prespeci- 
' fied objective while his peers observe the 

lesson for use of such instructional princi- 
ples as practice and feedback. A post- 
observation conference is held immediately 
after the lesson. Usually instruction t£dces 
place in a large arpa sUch as the oral arts 
room or the school cafeteria where the in- 
structor can monitor the entire process as- 
sisted by the master teachers. 

User Guidelines 

Based on our limited experience to uace, there ap- 
pear to be a few guidelines which might be of value to 



teacher educators considering the utilization of teaching 
performance tests. Some general suBgestjbns regarding 
the use of perfbim'ance tests are available ^elsewhere.** 
First, it has become apparent that the capability of 
the mini-lesson analyst (when group use of perform- 
ance tests is involved) is far more critical than had 
been anticipated. Unless the person supervising the 
analysis session is both convinced of the value o( the 
activity and able to provide instructional insights 
(when, for example, poor learner performance oc- 
curs), then the mini-Iessoh piay be far less profitable 
than possible. Too many teachers will write off mini- 
lessOT teaching efforts, particularly if poor perform* 
ahces occur; as unrepresentative of their performance 
in a real 'teaching situation. Thtis, on^of the supervi* 
sor's missions is to clarify the parallels of the mini- 
lesson activity and regular classroom teaching. Further, 
a deft instructional analysis can provide an unsuccessful 
mini-less(xi jeachef with a promise of future success 
during retaught or other subsequent lessons. The mini- 
lesson supervisor must eliminate the frustration that 
follows failure if no improvement plan is presented. 
Skilled supervisors are so important that, in the writer's 
view, unless they are available in sufficient numbers 
(or can be trained in time) group-type mini-lesson ac« 
tivities should not be undertaken. 

A second point relates to the use of mini-lessons , 
with children or adults. Reports to, date suggest that 
while a steady diet of mini-lessons for. young learners 
is palatable, exclusive use of adult learners (e.g., 
teacher educator classmates or colleagues> is less ac- 
- ceptable. Teacher educators who, for a variety of prac- 
tical reasons, may prefer to rely oh mini-lessbns %r 
adult learners, should strive to provide, as a change of 
pace, a few mini-lessons involving younger learners. 

Third, there seem to be some discernible dividends 
associated with providing some type of normative data, 
even roughly displayed, against which to interpret one's 
performance as a mini-lesson teacher. Without such 
comparative data, the teacher or supervisor is hard put 
to teil whether a given performance is good or bad. 
Referring back to table 2, it can be seen that there are 
clear diflferences, both inydifficulty and probable inter- 
. est, in certain of the tnint-lessons. How is the mini- 
lesson teacher to knowl whether his/hers was a well 
designed and executeiF lesson if no interpretive 
framework is provided? We need to supply mini-lesson 
teachers with what any golfer needs to make the game 

!n the teaching improvement kits distributed by Instruc- 
tionai Appraisal Services, Box 24821, Los Angeles, Calif. 
90024. 
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more meaningful, a rough notice of what constitutes a 
par performance." ^ 

Of course, normative data are not absolutely indis-. 
pensabic. for when a mini-'csson tij^acher discovers that 
his or her learners are all scoring around 50 percent 
on a task for which 90-100 percent proficiency had 
been anticipated, then the teacher at least has the 
knowledge that expectations have not been attained — 
and that's better than nothing. But comparative data 
are really a big help. 

Probleilii 

Teacher preformance tests, when employed as re- 
sources for teacher education evaluation or instruction, 
are in their infancy. Not surprisingly, therefore, a score 
of diaper-related difficulties have already arisen. And 
until a definitive Dr. Spock volume arrives to deal with 
our dilemmas, we'll have to do some trying and, unfor- 
tunately, some erring. 

One of the most basic problems facing wouldi^be 
*6sers of perfottnance tests stems from the instruments 
themselves. We have not yet made an acceptable effort 
to delineate the defining dimensions of performance 
tests, in terms of their content, objectives, posttest na- 
ture, background information, difficulty level, etc. Al- 
most all of the recently developed performance' tests 
have been devised more or less on the basis of expeii- 
ence and intuition. This situation needs to be rectified 
without delay. 

A related problem is the reliability of the teacher 
performance tests themselves. We do not yet have suf- 
ficient data to know how many mini-lessons a teacher 
must attempt before we can assess the teacher's overall 
level of competence on such tasks. It is expected that a 
teacher will not rate consistently high, say, on all 
mini-lesson attempts because his teaching score will de- 
pend in part on the subject matter of the lesson, in 
part on his own teaching approach to that lesson, and 
in part on how well other factors have been controlled. 
We are optimistic that teachers on the ends of the 
competency continuum can be isolated for special assist- 
ance or special commendation. 

An the absence of clearly posted signs indicating pars (per 
hole) of three, four, or five, the writer's early experiences on 
a golf- course would have suggested that an acceptable number 
of strokes per hole was something closer to 10, 15, or 20. 



A minor problem, but one which can yield trouble- 
some in-plementation difficulties, is the necessity to se- 
lect topics for mini-lcssons which will be viewed as 
important, both by the teachers who carry out the 
mini-lesson and by the students who are taught. In an 
effort to identify novel topics^ thereby eliminating the 
need for pretesting, a few topics have been chosen 
which are so esoteric as to yield atypical (or all too 
typical) student apathy. Perhaps it may be wiser to se- 
lect some main lin*;. curricular objectives and go to the 
trouble of pretesting prospective students in order to 
locate a suitably unknowledgeable learner group. Such 
mini-lesson topics might then be viewed as more mean- 
ingful by both teachers and learners. 

Finally, the logistics problems associated with proper 
use of performance tests shouldMje anticipated. An ex- 
amination pf the sitep-by-step details given in the ap- 
pendix will reveal the level of organizational planning 
needed to head off confusion. For instance, it is highly 
desirable to provide teachers of unsuccessful mini- 
lessons with an opportunity to replan and then reteacl} 
the lesson to a different group of learners. But provid^ 
ing these reteaching opportunities takes a good deal of 
planning time. Facied with such planning frustrations, 
many teacher educators will be tempted to return to 
the less tajing, but perhaps less effective, lecture-dis- 
cussion classroom format. Anticipating logistical prob- 
lems can help avoid them. A competent secretary or 
teaching assistant ^can alleviate logistical distractions by. 
working out the organizational requirements in ad- 
vance. 

Review 

In retrospect, an attempt has been made in this^ 
paper to discuss possible applications of teaching per- 
formance tests to the activities of teacher educators. If 
a bias in favor of such applications was reflected in the 
paper, this was only natural, for such is the writer's 
bias. Although still a rather primitive tool, the per- 
formance test may be a valuable instrument to teacher 
educators. As we look at ancient man's hand axes, we 
may view them as ihcredibly simple devices, yet their 
impact was enormous. No strict analogy is being pro- 
posed here, only a plea to consider teaching perform- 
ance tests as an additional tool in our teacher educa- 
tion kits. Think of how many sabre-toothed tigers we 
might slay. 
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tary Principal, Vol. 52, No. 5, February 1973.< 
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TEAM S MINI-LESSON 

CLINIC GUIDELINES 
UCLA Teacher Education Laboratory 
Fall 1972 



Commencing with the second week of the quarter, 
each student enrolled in team S will be required to 
partkipate either as an instructor or a learner in a 
weekly^ mini-lesson clinic. The procedural elements of 
that pa^icipation will be outlined below. 

\ 

\ 

One Booklet Required 

During the first week of the quarter, when enroll- 
ments hive settled down to some extent, a mini-lesson 
clinic assignment list will be distributed. Nine students 
will be assigned to a particular mini-lesson clinic 
group. Three of the students in each group will be des- 
ignated' as the group, three as the B group, -^nd 
three as the C group. (These arbitrary designations do 
not reflect the instructors' grading vision and will have 
no relationship to the final grade earned in the 
course.) Each student should, without delay, then go 
to the student bookstore and purchase one copy of the 



adult form Teaching Improvement Kit, which beare the 
same letter as the one he or she was assi^ed. For ex- 
ample, a student in mini-lesson clinic No. 13 who is 
designated as a member of the B group should pur-, 
chase a copy of Form B of the Adult Teaching Im- 
provement Kit. Please note that there are some optional 
Teaching Improvement Kits available in the bookstore 
for use with children, but that the required Teaching 
Impto 'ement Kit, one of the three, forms available 
(i.e., A. B, C), is an' adult kit. With these materials in 
hand, the mini-lesson clinics will get underway the sec- 
ond week of the course in the assigned rooms. 

One mini-lesson will be taught at each clinic, thus 
the individual assigned to teach on a given date must 
be present and prepared to teach. Failure to do so will 
result in a severe grade penalty. Yet, excused absencbs 
will occur. Therefore, all students should be ready to 
teach one week early in case of an unanticipated ab- 
sence by the reguhriy assigned teacher. 
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General Nature of the Participation 

To provide a brief overview, an individual student*s 
responsibilities will be described. V\r$U a student will 
be obliged to instruct the six students in his mini*lesson 
clinic group who have been assigned other letters than 
his/her own. For example, a C member of the mini-les- 
son group wtiuld. be obliged to, teach a lesson from the 

Teaching Improvement Kit to^he^^six A aod B stu- 
dents in the group. Second^^each student must aCt as a 
.critic for two mini-lessons taught by the members»of 
the mini-lesson group who possess the same letters. To 
illustrate, a B member of mini-lesson clinic group No. 
7 would not only teach one B mini-lesson him/herself, 
but would serve (along with another B member) as a 
critic while two other B lessons are being taught to the 
six A and C students. Finally, each student wjll be 
obliged to locate one or more groups of at least mfee 
adults to whom each of the three mini-lessons in his or 
her teaching kit can be taught, using a form of the 
posttcst other than that employed in the mini-lesson 
clinic. Thus, in summary, each student will teach one 
mH)f4esson to team S classmates, critique two other 
mini-lessons being taught to team S classmates, teach 
three mini-lessons to adults other than team S class- 
mates, and serve as a learner for six mini-lessons. 

Procedural Specifics 

Now, in more detail, here are the step-by-step pro- 
cedures to be followed by each student. First, consult 
the mini-lesson assignment sheet to note which group, 
letter, and which week you have been assigned. Now 
read the first thrqc chapters of ihe Teaching Improve- 
ment Kit (pp. I- 15). You may wish to examine the 
mini-lesson assignment linforraation in chapter 4. Do 
not examine the posttests (on greep and blue sheets) 
which are included in the Teaching Improvement Kits. 
The mini-lesson clinic sessions have not been assigned 
to influence your grade in the course, but are specifi- 
cally intended to hel^ you improve your instructional 
skills. As a consequertoe. cxaihination of the posttests 
prior to your teaching the lesson would reduce the 
likelihood that the mini-lesson would be beneficial to 
you. 

On the day designated on the assignment sheet, you 
will be obliged to teach one of the three lessons in 
your improvement kit (1, 2, or 3). On that day you 
will instruct (for no more than 15 minutes) the six 
members of your group who have been assigned other 
letters; e.g.. for the B students this would be the A and 
C students. The two members of your group who have 



been assigned thf same letter will sit somewhat apart 
from the student group and will attempt to analyze the 
quality of your teaching plans, activities, etc. At the 
conclusion of the 1 5-minute lesson, the perforated cop- 
ies of post test I (green paper) should be removed 
from your Teaching Improvement Kits and distributed 
to your six students. You should not have seen the post- 
test prior to this moment. These tests should be 
quickly completed and scored by the mini*Iesst)n clinic 
group leader. Qn the basis, of 'th^ average scores on 
(I) the interest rating and (2) the posttest, discussion 
of the teaching should be conducted in terms of the re* 
suits produced. That is, good learner performance 
should result in a discussion focused on instructional 
tactics which seemed effective; poor resuhs should lead 
to a discussion focused on instructional procedures 
which might be altered. To assist you in judging how 
directive a given teaching performance was, at the rear 
of each Teaching Improvement Kit data are available 
regarding how successful other teachers have been with 
each mini-lesson. Results of each student^s perform- 
ance will be turned in, but not for yading purposes, 
by the mini-tesson clinic leader. 

On two occasions, therefore, each student will be 
serving not as a learner but as a critic. On those two 
occasions the student (having access to the Teaching 
Improvement Kit under consideration) will ^e had 
an opportunity to examine the mini-lesson assignment 
prior to the clinic. He or she will undoubtedly have 
some thoughts regarding an appropriate instructional 
procedure. This may be beneficial during the post- 
lesson analysis session. It will be useful to have two peo- 
ple, other than the clinic leader, who are as conversant 
with the mini-lesson requirements as the teacher. Dur- 
ing the analysis discussion, it is anticipated that the 
mini-lesson clinic leader and the two critics will take 
primary responsibility for isolating elements of the les- 
son that were particulariy effective or ineffective. Re- 
member that the appraisal of instructional means 
should be made chiefly in tern^s of learner results; that 
is. the averaged interest ratings ahd posttest scores. 

The final responsibility for each team S student is to 
teach all three of the mini-lessons in his or her kit to 
another group of adults; that is, someone other than 
team S students. Since each of the mini-lessons in the 
Teaching Improvement Kits has two equivalent post- 
tests, the team S students should not, either delibf^r- 
ately or inadvertently, examine posttest 2 (Wue 
sheets). Ideally after the original teaching of that ^ 
mini-lesson in the mini-tesson clinic, each student 
should locate a group of at least three adults (friends, 
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relatives, or people off the street) who would be will- 
ing ti> serve as students for the approximately 20 min- 

• utes involved fur this assignment ( 1 5 minutes of teach- 
ing plus 5 or so minutes of resting). The mini-lesson 
should be taught and results summarized on the Mini- 
lesson Postlest 2 Summary Report Forms which have 
been provided j[see attachment). The information 
called for is brief, yet should be completed in its en- 
tirety. These Mini-lesson Posttest 2 Report Forms 
should be turn;:d in weekly during the 9 o'clock class 
meetings. Iri other words, an individual team S stu- 
dent should liave an opportunity to reteach the mini- 

' lesson he originally taught to team S students after a 
critique of the first lessen. Hopefully, this analysis will 
be useful in promoting improved learner performance. 
The other two lessons in the kit will, for that student, 
be taught for the first time. Ideally, the discussion of a 
classmate's teaching of that same lesson will prove use- 
ful in helping devise an effective lesson. 

Other Considerations 

Results of the weekly mini-lesson clinics will be 
made available so that students who wish to compare 



their performance with that of other team S members 
may do so. As indicated previously, there are other, 
optional Teaching Irnprovement Kits available in the 
bookstore for use with younger learners (elementary 
school age). Furthermore, mini-lessons in the rfdult 
kits can be used with mature secondary school stu- 
dents. Thus, it is possible to teach mini-lessons from 
another kit (other than the letter assigned) to high 
school age students. Either of these activities; that is, 
teaching mini-lessons to elementary or secoi^^ 
school youngsters, should be considered optiona|r for 
team S students. Remember, the whole purpose of the 
mmi-lesson clinic operation is to improve the team S 
member's skill in accomplishing prespecififed objectives 
with teaching procedures which are also interesting to 
the learners. The amount of time that yoii can devote 
to promotion of this particular competency will un- 
doubtedly yield great benefits to your future students. 
As a consequence, please approach the mini-lesson 
clinics and outside mini-lesson assignments as a real 
opportunity to increase your instructional skills. 



MINI-LESSON POSTTEST 2 REPORT FORM 

Your name Mini-lesson 

Date mini-lesson taught _^ Today's date 

' Setting for teaching and types of learners (one or two sentences) 



Number of students taught 

Results: average interest rating \ 

average percent correct on posttest \^ 

Was this the second time you taught the mini-lesson? Ve$ ^ ■ ■ No 

If this was the second time, suppl37^he average interest ralnijg . 

and average percent correct . for the first tune you taught the 

mini-lesson. 

Comments (optional): ■ ^- 



Below give the names and phone numbers (if available) of at least three of the 
students you taught. 
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Identification And Assessment Of Minimal Competencies 
For Objectives-Oriented Teacher i^ducation Programs* 



By 

W. JAMES POPHAM 



Classroom teachere have hundreds of things to do. It 
should follow, then, that prospective classroom teach- 
ers have hundreds of things to learn. But top many 
well meaning teacher educators have used these two 
premises to draw the conclusion that they therefore 
have hundreds of things they must teach, and that rep- 
resents a serious error. 

Given the instructional time available in typical 
teacher education programs, we must be more modest 
in our aspirations. When we ask teacher education can- 
didates to swing an axe at every tree in the forest, they 
may fail to fell even a sapling. Far too many teacher 
education programs are predicated on a cover-the- 
waterfront concept: that is, give 'em the works in cul- 
tural foundations, educational psychology, and instruc- 
tional methods. And the use of the verb "cover*' is 
quite deliberate. Most teacher educators feel compelled 
to cover content in their courses that they perceive as 
germane to the teacher's responsibilities. When these 
professors have covered such content, they sleep easier 
at night. Few of these coverage-culprits ever verify 
whether their extensive coverage of subject matter ever 
results in any payoff for the teacher candidate, other 
than the ability to pass a memory-oriented final exami- 
nation. And as in so many content-coverage courses, 
not just those in a teacher education sequence, what 
was covered one semester has faded from the student's 
memory by the first week of the next term. 

Particularly at a time when teachers are being 
weighed more scrupulously on the public's accountabil- 
ity scales, teacher educators who persist in covering all 
relcvent topics are probably doing an injustice to the 
teachers i they are responsible for preparing. It makes 
more sense for the teacher educator to select a limited 
number of competencies which teachers should ac- 
quire, then focus the program's resources on making 
certain these skills are acquired. 

Now even if this point of view (i.e., a focus on the 
attainment of a modest set of competencies) were as- 



siduously followed, there would still be considerable 
disagreement regarding which competencies to promote 
or, in more general terms, what kinds of content to 
emphasize. Some would prefer to focus on the teach- 
er's attainment of a wide repertoire of instructional 
technique. Others might attend more directly to *i5ie 
teacher's becoming a more integrated human being. 
Still others woulif emphasize the tocher's acquisition 
of subject matter expertise. The alternative emphases 
are myriad. 

o 

An Objectives Orientation 

The remainder of this analysis will describe a partio* 
ular orientation which can be described in genera! 
terms as an outcomes-focused 'approach. An outr 
comes-focused approach emphasizes the results that a 
teacher's efforts produce in modifying the behaviors of 
learners and can be contrasted with more process- 
focused strategies which attend to the instructional 
ploys a teacher utilizes with pupils. Because instruc- 
tional objectives can serve as a convenient way of de- 
scribing the intended results a teacher wishes to 
achieve with learners, we may refer to one variant of 
an outcomes-focused approach to teacher education as 
objectives-oriented. An objectives-oriented strategy for 
educating teachei's will be treated here. 

The rationale for an objectives-oriented approach to 
teacher education characteristically rests on a central 
assumption; namely, that the raison d'itre for a class- 
room teacher is to bring about worthwhile changes in 
learners, i.e., important kinds of improvements in their 
knowledge, attitudes, skills, etc. Proponents of an 
objectives-oriented teacher education program believe 
that even if a teacher lectures with consummate skill, but 
the students are left unchanged, the teacher has failed. 



♦ A paper presented at the annual meeting of the American 
Educational Research Association, New Orleans, February 26- 
March 1, 1973. 
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Similarly, they contend that even if the teacher has led 
a nondirectivc discussion with the artistry of Carl Ro- 
gers, but the students are basically unaflfected. then the 
teacher has failed. The criterion, quite clearly, is not 
what the teacher does, but what happens to pupils as a 
consequence of what the teacher does. Few objectives- 
oriented t<5acher education programs are not somehow 
wedded to this basic view of a teacher's mission. 

But how do objectives enter the picture? Well, their 
chief value is In helping teachers identify more clearly, 
prior to instruction, the kinds of changes whldi should 
be promoted in the learners. Statements of Instructional 
objectives are nothing more than that, convenient de- 
scriptors of intended changes in learners. In the early 
1960*s, advocates of the oft-maligned behavioral objec- 
tive endorsed such formuUtions vigorously because of 
their^ focus on the learner's post-Instruction behavior, 

* not on what the teacher w^^^oftig to do or the contents 
<hat^ the course would coverj It is unfortunate that 
some educators have become so entan^ed with behav- 
ioral objectives they have made them a fetish. Precise 
instructional objectives, in the main, are simply state- 

" ments of what teachers want to happen to learners as a 
result of instruction. The more explicitly these Inten- 
tions can be formulated, the better we can tell whether 
the intentions have'been realized, and it Is for that rea- 
son that most proponents of. objectives strive for 
measurability as the sine qua non of an acceptable 
objective. But remember the central purpose of an in- 
structional objective — it is to help an instructional 
planner conceptualize the kind of changes to be pro- 
moted in learners. 

Proponents of instructional strategies featuring meas- 
urable objectives should forthrightly admit that their 
conception of the instructional process is generally one 
based on rational decision making. Some critics of an 
objectives-oriented approach denigrate such strategies 
as ^industrial models'* of education and therefore 
somehow unworthy of man's truly humanistic capabili- 
ties. They would prefer less systematic and Intellectual- 
ized approaches, favoring Instead more intuitive, dy- 
namic models. But when Aristotle isolated the essence 
of man as his rational animality and held that a per- 
son's potentials were realized to the extent that those 
rational powers were actualized, he offered objectives- 
oriented teacher educators a satisfactory counter-argu- 
ment. To plan one's actions on the basis of the action's 
likely consequences is less industrial than it is rational. 
To be clearheaded is not to be mechanistic. To define 
anticipated outcomes in advance docs not relegate one 
to an assembly line mentality. On the contrary, to be 



rational in our education decisions will give our stu- 
dents the best chance of prospering from the education 
we provide them. 

Are Instructional Techniques Unimportant? 

With most objecilves-oriented teacher education pro- 
grams, it is proper to assert that a distinction is drawn 
between instructional means and instructional ends, 
with the stress typically on ends. But as anyone who 
has attempted to achieve a significant end will agree, it 
is brpught about by employing appropriate means. 
Hence, an outcomes-focused teacher educator must be 
particularly attentive to instructional techniques, en- 
hancing the teacher's skill In employing a wide reper- 
toire of teaching tactics, for it is only through the. judi- 
cious use of such procedures that significant kin0s of 
results in learners ddn be attained. 1 

Minimum Competencies^ _ / 1 

Programed instruction specialists are familiar ^th 
an approach to the development of instructional m|^te- 
rials known as lean programing. In su^h a strategy ^he 
programcr tries to accomplish a given instructional 
objective with the least possible athount of instructional 
stimulus material. Aside from its obvious economic ad- 
vantages, lean programing cjinies with it a dividend 
when an early version program is unsuccessful. Ibis 
easier to improve a low density program bjjsupple- 
mcnling it. than it is to delete segments of a mgn den- 
sity program, for in the latter approach we may be ex- 
cising the very ingredients that contributed to whatever 
effectiveness the program possessed. 

Similarly, what is being proposed here may be 
described as lean competency promotion, for only 
three competencies of an objectives-oriented teacher 
education program will be recommended. Now surely 
teacher education candidates will learn other things as 
they complete their preparation programs; they may 
even learn some of the hundreds of things referred to 
at the outset of this paper. Since it will be easier to 
supplement a few minimal competencies than to delete 
from a more diverse array, it is proposed that only 
three such skills be emphasized in an objectives-ori- 
ented teacher education program. 

The remainder of this discussion will isolate these 
ihree minimal competencies, offer some support for 
tneir importance, and describe alternative methods of 
assessittg the degree *to which each has been attained. 
These three competencies may be used as the guiding 
goals of either a preservice or inservice teacher educa- 
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tlon effort. The differences in strategics for promoting 
the competencies for experienced or beginning feeders 
are only superficial and the differences in assessment 
tactics ahnost nonexistent. 

Competency Number One 

Since the chief assumption of an objectives-oriented 
program is that teachers should promote worthwhile 
changes in learners, it is not surprising that the initial 
competency to be fostered deals with that basic skill: 

1 . Teachers must be able to achieve prespecified 
instructional objectives with diverse kinds of 
learners. 

This competency implies that ^ skilled teacher 
> should, when presented with clear statements of in- 
tended changes in learners, oe able to jjevise instruc- 
tional sequences which will work; that is, which will 
bring ai^ixt the sought-for changes in the learners. 
Further, the cpmpetency indicates that this skHl be 
manifest with different kinds of learners; for example, 
children of differing ages, ability levels, ethnic back- 
grounds, socioeconomic status, etc. 

The truly professional teacher not only will need to 
be conversant with tested instructional principles in 
order to design such instructional plans but will have 
to discover what kinds of teaching tactics personally * 
prove effective. Not all violinists can get good music 
from the same fiddle. Different people must adopt dif- . 
ferent teaching styles. For some teachers a nondirective 
approach will work beautifully, while for other teachers 
such a strategy would be a disaster. It is imperative 
that a teacher discover what communication style, cou-* 
pled with relevant instructional principles, typically re- 
sults in the attainment of prespecified instructional 
objectives for that teacher. 

Assessment Tactics 

There are two (M'ime contenders for assessing the de- 
gree to which this initial competency has been attained. ' 
The first of these involves the use of teaching perform- 
ance tests (or instructional mini-lessons) whose ration- 
ale and applications are described elsewhere.* Briefly, 

» W. James Popham, ^'Performance Tests of Teaching Profi- 
ciency: Rationale, Development, and Validation,** American 
Educational Research Journai^^January 1971, 8{ 1 ), pp. 105-117; 
W» James Popham, Applications of Teaching Performance 
Tests to Inservtce and Preservtce Teacher Education. Paper 
presented at the annual meeting of the American Educational 
Research Association, New Orleans, February 26-March I, 
1973. 



a teacher is given a measurable instructional objective 
(typically dealing with a novel topic) along with any 
necessary background information needed to under- 
stand the objective. The teacher plans a short lesson 
(c.g„ 15-20 minutes) designe.d (1) to accomplish tlie 
objective and (2^' to be interesting to the learners. The 
lesson is then taught, either to a small group of six to^ 
eight learners or to an entire class* At the conclusion ' 
of the lesson, a posttest is adm|gjstered to the learners. 
The posttest has not been previously seen by the 
teacher,^ but its nature is readily inferable from the 
clearly stated objective. A form requesting the learner 
to **rate how interesting the lesson was** is also pro- 
vided. The teacher is judged on the basis of whether 
both the cognitive intention (the objective as measured 
by the posttest) and the affective intention (the pro- 
motion of learner interest as measured by the rating 
form) have been achieved. 

The recency of serious research attention given to 
teaching performance tests as an evaluative tool proba- 
-biy renders them inappropriate at the present titoe for 
assessing individuals other than those at the extremes 
of a distribution; i.e., the particularly good or particu- 
larly pioor goal achievers. Performance tests may also 
be used to evaluate the efficacy of a teacher Education 
program by administering them on a preprojgram and 
postprogram basis to the teachers involved. For ex- 
ample, suppose two teaclhing performance tests (X and 
Y) were employed. One of each could be administered 
to half the teachers (or teacher candidates) at the 
beginning and at the close of the program. The pre- 
diction would be that X^re < 5Cp„Hi and Y^r. < Ypoai- 

Although the reliability of different teaching per- 
formance tests has not yet been established With suffi- 
cient precision to warrant their use for Individual eval- 
uation, with more systematic delineation of the key 
elements constituting such tests we may find that in the 
future they can be used for more fine-grained analyses 
of individual teacher's mastery of competency number 
one. 

A second approach to the assessment of the initial ^ 
competency is to allow teachers to posit their own in- 
structiciial objectives, develop a congruent mastery ex- 
amination« then instruct a group of learners in order to 
attain the objective. Interest ratings can also be em- 
ployed here. Because an objective generated by a 
teacher can be less readily compared with objectives 
pursued by other teachers, there is the additional re- 
sponsibility of the educator to appraise the quality of 
the teacher's objective, not to mention the consonance 
of the test wif Ii ^the objective. The advantage of this 
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second approach is that the teachers do most of the 
work in generating the objectives, tests, etc. Further, 
because the topic need not be novel, the objective may 
be deagned for longer periods of instructional time as 
part of the ongoing curriculum activities. With topics 
which fall within the learner's probable experience 
base, a pretest must be administered to establish entry 
behavior level. 

There pre, of course, a number of en route skills 
which a teacher should master on the way to attaining 
this initial competency, but by employing one, or possi- 
bly both, of these assessment tactics the teacher educa- 
tor should be able to determine whether competency 
number one has been satisfactorily promoted. 

Competency Number Two / , 

It has been observed elsewhere that one of the con- 
soling features of conventional^ instruction is that it is 
characteristically so impotent we need not worry too 
much about what its goals Similarly, if a teacher is 
not particularly proficient at accomjdtshing instruc- 
tiotal objectives, then we need not be too concerned 
about what the teacher is attempting to do. Just sup- 
pose that a teacher has achieved competency number 
one, that is. has become skilled in promoting the learn- 
ers' accomplishment of prcspecified objectives— then it 
becomes extremely important to have the teacher direct 
this instructional prowess toward the proper goals. Ac- 
cordingly, the second minimal competency of an objec- 
tives-oriented teacher education program becomes: 

2. Teachers must be able to both select and 
generate defensible instructional objectives. 

Since teachers who are skilled goal-achievers must 
become able to either generate or select worthy goals, 
it is fortunate that curriculum specialists are finally dis- 
carding their customary intuitive approaches in favor 
of more practical goal-determination procedures. For 
example, the current refinement of large scale educa- 
tional needs assessment approaches can be translated 
into practical guidelines for teachers who wish to de- 
termine educational objectives in a more rational fash- 
ion. Screening of goals by the use of various taxono- 
mies of educational objectives, such as those devbed by 
Gagn^, Mechner, Bloom, and Krathwohl, also can lead 
to the adoption of more appropriate goals. Without 
going into those technical procedures more intensively, 
it can be established that these are schemes now avail- 



able which, albeit imperfect, can aid a teacher in the 
attainment of competency number two. 

Since there are now available to teachers an increas- 
ing number of extant pools of instructional objectives, 
thereby permitting teachers to select objectives rather 
than be obliged to generate them personally, it seems 
wise to develop the teacher's proficiency in objectives 
selection as well as objectives generation. 

Assessment Tactics 

There are several procedures available for assessing 
the teacher's mastery of competency number two. On© 
procedure would require teachers to generate a set of 
measurable objectives, then have these judged by oth- 
ers (using criteria of significance, suitability for learn- 
ers, etc.). A description of real or fictitious learners 
could be given as part of the goal-generating task, and 
then descriptions could be examined by judges pfior to 
the appraisal of the goals. Teachers could be asked to 
generate such objectives at the be^nning of the teacher 
education program and at its conclusion. These papers 
could be coded and rated by judges without knowing 
the time at which the papers had been written. The 
prediction, of course, is that the end-of-course objec- 
tive would be rated hi|^er. 

Another approach' to assessment might cdnsist of 
having a teacher select a specified number of objectives 
from a larger pool of such objectives, then have the 
selections appraised by others. As with the previous as- 
sessment approach, subsequent judgment of the teach- 
er's objectives (either generated or selected) can be 
rendered according to very general or very specific cri- 
teria. 

Additional assessment tactics might involve the 
teacher's describing, in an exam-like setting, alternative 
procedures for selecting or generating defensible objec- 
tives. These descriptions, as with the^ first two assess- 
ment tactics, might then be evaluated by judges, and if 
desired, on » preprogram and postprpgram basis. 

Variations of these approaches are possible, of 
course, such as having teachers themselves rate the ad- 
equacy of objectives selected by other teachers, sucK 
ratings being subsequently appraised. 

Competency Number Three 

The first two competencies have been highly related 
to instructional objectives, their determination and ac- 
complishment. The third minimal competency of an 
objectives-oriented teacher education program is, un- 



55 



like the first two, quite unrelated to objectives. In fact, 
it is almost antithetical to a concern about objectives: 

3. Teachers mim be able to detect the unanticipated 
effects of their instruction. 

In spite of good intentions, even combined with 
good Jntcntion-achieving skills, a teacher's efforts will 
often produce unforeseen detrimental and beneficial re- 
sults with students. Hence, all of the outcomes of in- 
struction must be considered. Thp teacher must be 
skilled in -determining the totality of what happened to 
students, including of course what was supposed to 
happen. 

There arc several different techniques a teacher 
might employ to discern unanticipated effects of in- 
struction, such as the use of (a) relatively unstructured 
anonymous student questionnaires (e.g., **List the best 
and worst things that happened to you because of this 
course."), (b) struc^red anonymous questionnaires 
which attempt to isolate the positive and negative ef-' 
fects, cognitive as well as effective, which might occur 
because of instruction, (c) quasi-projectivc techniques 
such as the assignment of an essay to the class dealing 
with topics such as "My reactions to Biology I" or 
**Autobiogr^hy of a U.S. History Student," and (d) 
the investigjption of the results of a tetfther*s efforts by 
a colleague) who follows Scriven*s Coal Free Evalua- 
tion strateg^; that is. Who attempts to discover (with- 
out even knowing what the teacher's objectives were) 
what happened to the students. 

It is imperative that objectives-Oriented teacher edu- 
cation programs promote this third competency, for 
without it there is too much danger that teachers may 
have marvelous intensions, accomplish them beauti- 
fully, but at the same time promote harmful side effects 
which more than cancel out the anticipated results. 

Assessment Tactics 

Even more clearly than the first two competencies, 
this third is heavily dispositional in nature; that is, we 
must strengthen teachers' dispositions to attend to the 
unanticipated effects of instruction. 

One rather primitive method of getting at this dispo- 
sition is to employ an inventory such as that presented 
in the appendix. The rationale and scoring scheme of 
this inventory, *ix>oking at Teaching," is supplied 
along with the inventory. In brief, a student is asked to 



register various degrees of agreement with a series of 
statements regarding instruction, some of which deal 
with the use of unanticipated side effects. 

On the skill side of this competency, we could al- 
ways ask a teacher to describe as many ways as possi- 
ble whereby a teacher who wishes to can detect such 
effects. 

Perhaps simulation approaches offer the greatest 
promise with respect to ascertaining whether this c(mi- 
petency has been mastered. Instructional situations 
could be presented to the teacher, either on paper, vi- 
deotape, or film, in which there are clearly intended 
objectives plus some evidence as to the' degree to 
which they had been achieved. In addition, there 
would be some subtly identified unanticipated effects of 
instruction. The teacher would-be asked to evaluate the 
worth of the instruction, and a record would be made 
of the extent to which attention to the unanticipated 
side effects had been incorporated in that evaluation. 

GeUing teachers to describe their general evaluation 
strategies is another altemative> for one could then in- 
spect such descriptions to see if, in response to this 
largely unstructured stimulus, unanticipated side effects 
were built into the teacher's analysis plan. 

A Beginning 

In review, an attempt ^as made in this analysis to 
defend the proposition that, fewer competencies shotdd 
be used as the organizing dtrdcture for teacher educa- 
tion programs. An objectives-oriented teacher' educa- 
tion-iipproach was ^described and three minimal compe- 
* tencies for such a strategy were isolated, along with 
alternative assessment tactics for each. , 

These three competencies were identified on the 
basis of the writer's experience with outcomes-focused 
teacher education programs. They are predicated on 
the Belief that teachers who possess such skills will be 
able to do a better job for the learners they attempt to 
serve. / 

The assessment tactics, however, are certainly not as 
sophisticated as one would wish. Hopefully, this deline- 
ation ^of possible assessment ploys may stimulate other 
objectives-oriented teacher educators to share their pet 
assessment devices. More importantly, perhaps, it may 
encourage teacher educators, both objectives-oriented 
and those of other persuasions, to scrutinize the ade- 
quacy of minima] skills offered by their programs and 
the schemes which they employ for their assessment. 
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Appendix , 

« 



Looking at Teaching 



Directions. 

This inventory consists of four brief descriptions of 
instructional situations, each of which is followed by 
five statements. Please register the extent to which you 
agree or disagree with each of the five statements by 
circling the appropriate letters to the left of each state- 
ment according to the following scheme: 

SA «^ Strongly Agree 

A Agree 

U « Uncertain 

p » Disagree 
Sb » Strongly Disagree 

There are no right or wrong answers to this inven- 
tory. It represents an effort to secure your reactions to 
various views of instruction. Therefore, please be as 
candid as possible in your responses. 

Situation I. 

Mr. Hill is a junior high school history teacher who 
believes very strongly in **open education.*' He designs 
class sessions so that they are relatively unstructured, 
with a heavy emphasis on, discussions plus individual 
reports of resource projects students have initiated be- 
cause of their personal interests. Mr. Hill finds that 
students are generally responsive to his approach, but 
some of them register dissatisfaction that they are not 
learning enough to prepare them for serious high 
school history classes. 

SA AU D SD 1. Mr. Hill has no right to emphasize 
*opcii education if it deprives students 
of standard course coverage* 

SA A U D SD 2. The type of instruction Mr. HHl is 
providing will generally be Uninterest- 
ing to students. 

SAAUDSD 3. It is impossible to combine any 
form of open education with adequate 
content coverage. 

SAAUDSD 4. Mr. Hill should have devised ex- 
plicit instructional plans, almost day- 
by-day details, prior to the beginning of 
the semester. 



SAAUDSD 5. Mr. Hill should not try to detect 
any effects of his instructional scheme 
other than those he guessed mig^t 
emerge. 

Situation II. ' 

An elementary school teacher, Mrs. Price, usually 
works with third- or fourth-grade children. Normally, 
most students who come^to her class cm read quite 
well, but 20-25 percent cannot. She devises special 
self-instruction learning centers for these pocMr readers 
and encourages them to go to the centers during un- 
scheduled class time so that they can improve their 
reading abilities. Although the performance of these 
children Indicates they have become somewhat better 
readers, they are subjected to considerable verbal, 
abuse by the good readers in the class whenever they 
participate in the learning centers. 

SA A U D SD 6. Self-instructicmal materials can be a 
valuable resource for any teacher. < 

SA A U D SD 7. Mrs. Price should have done noth- 
ing special for the poor readers coming 
to her class because their deficiencies 
were the resp<Hisibility of previous 
teachers. 

SA A U D SD 8. Even though it was not foreseen^ 
Mrs. Pri(Se should realize that the neg- 
ative effects of the ^se they received 
may have been more harmful to the 
poor readers than whatever progress 
they maue in reading. 

SA A U D SD 9. It is normal for 20-25 percent of 
children to read badly; so any gains 
Mrs. Price can get will be all the more 
valuable. 

SA A U D SD 10. Poor achievers must always expect 
to experience a certain amount of de- 
risiveness from normal and high 
achievers. 

Situation III. 

Mr. Cohen is a high school English teacher who 
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.plans his instruction with inordinate care. Prior to each 
class he details every significant level of achievement 
he believes students should make as a consequence of 
his course. He also attempts to spell out any major at* 
titudinal or interest shifts he is attempting to promote 
with the pupils. At the close of the academic year he 
evaluated his English class totally in terms of whether 
these intended changes, both intellectual and attitudi- 
nalt have been produced in the learners. 

SA A U D SD 1 1 . Mr. Cohen should certainly deter- 
mine whether, at the beginning of the 
academic yearjiis pupils can always 
display the intended behaviors. 

^SAAUDSD 12. If learners are informed of the 
clear expectations of an instructor, such 
as those which Mr. Cohen appears to 
have, they will tend to be less anxious 
about the learning situation. 

SA A U D SD 13. Beyond the* clearly delineated be- 
havioral changes which Mr. Cohen has 
identified, he should discern whether 
there were any adverse or beneficial 
effects of students which he had not 
considered prior to. instruction. 

SA AUDSD 14. Mr. Cohen's careful planning, al- 
though commendable in the abstract, 
will probably take too. much valuable 
energy from his actual instruction. 

SA A U D SD 15. In general, humanity subject fields 
such as English are the least amenable 
to an instructional approach dependent 
in the prespecification of educational 
goals. 

Situatiqn IV. 

Ms. Harold is an elementary schooPs music instruc- 
tor who must work instructionally with children at all 
levels. She feels terribly overburdened with the number 
of youngsters she is obliged to service, t^us devises ex- 
tremely intensified music lessons for each grade level. 
Although there is little doubt that the children arc 
learning about music, there are a number of indications 
that they are becoming antagonistic to music in the 
process. Ms. Harold behaves as though she were Obliv- 
ious of these negative attitudes. * 

SA AUDSD 16. Ms. Harold is probably required 
to undertake instruction beyond what 
might be expected of a typical teachen 
hence wc should excuse any negative 
attitudes she might be creating. 



SA A U D SD 17. The negative attitudes the children 
are developing are relatively unimpor- 
tant, particularly because the children 
are learning a great deal about music. 

SA AUDSD 18. Only in esthetic fields such as 
music and art will intensified instruc- 
tion lead to student negativism. 

SA AUDSD 19. Ms. Harold should abandon any 
. emphasis on music skills and focus in- 
stead on promoting positive attitudes 
toward music. 

SA AUDSD 20. Ms. Harold must recognize that 
unanticipated effects of ^instruction are 
potentially more important tfean in-' 
tended effects and should strive to iden** 
tify such effects as the negative atti- 
tudes seen here. . 

Scoring Direction^ 

LoQking at Teachiiig 

This inventory is designed to detect how predis* 
posed teachers are to detecing the unanticipated effects 
of instruction and use them in evaluating the quality of 
instruction. Pf the 20 statements with \vhj,:h respond- 
ents are to indicate agreement or disagreement, only 
five deal with thi> question. The other 15 items are in- 
cluded only to camouflage the real purpose of the in- 
ventory so. that respondents are not readily able to de- 
tect the scKially desirable way to answer the items. 

The. five items and the scores associated with each 
response are given below. Omitted items should be 
given a score of 3. 



Item 



KEY . 

Points 



SA 


A 




D 


SD 


I 


2 


3 


4 


5 


5 


. 4 


3 


2 


1 


5 


- 4 


3 


2 ' 


1 


1 


2 ' 


3 


4 


5 


5 


4 


3 


2 


I 



5 

8 
13 
1.7 

zo 

Since a person might earn a maximum of 25 points 
on the basis of these five items, scores approximating 
25 should be considered to reflect a predisposition to 
consider unanticipated side effects important in evalu- 
ating the quality of a teachers instructional efforts. 
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As experience with performance-based teacher edu- 
cation has accuiT^lated, the interrelated problems of 
competency definition and competency assessment have 
come increasingly into focus. On the one hand it i^^JI 
recognized that teaching competence is something more 
than the mastery of knowledge and simple teaching 
skiPs or behaviors, Sut on the other it is recognized 
that as soon as thji:^hition of competency extends 
beyond the knowledge ai^ skill level, the matter of as- ' 
sessment becomes inordinately complex- In fact, in the 
eyes of many, it takes on properties that demand ftiore 
from the technology of measurement, and evaluation 
than that technology has at the monjent to give: 

As a result of this circumstance the designers of 
teacher education programs face a difficult choidfe. If a 
teacher ^education .program is to be performance- or 
competency-based, and not be a mockery of those con-, 
cepts, provision must be made for the collection of ac- 
ceptable evidence of competency demonstration. Yet 
the technology of assessment is such that when teaching . 
competency is defined in terms of performance in 
ongoing school settings, or in terms of the outcomes 
expected to be achieved through teaching (Tiimer's 
criterion levels 3, 2, and 1), the wherewithal! to meas- 
ure such outcomes simply is not availablcMWhen con-" 
fronted with this circumstance most program designers 
adopt the simpler definitionsv^of teaching competency 
and proceed with program development as if such defi- 
nitions were acceptable. ^ 

This circumstance, in our opinion, helps account for 
the large number of teacher education programs in ex- 
istence today that label themselves as being compe- 
tency- or performance-based, but choose to define 



» This paper was presented in outline form at the meetings 
of the Multi-State Consortium on Performance-Based Fcacher 
Education, New Orleans, February 25-28, 1973. 



"competency" at the knowlcfdge or siipple skill level. 
Since sq few. institutions have moved to implemeiU a 
performance-based teacher education program that is 
ba^d upon higjtidc order definitions of competency, 
and those that have moved in this direction have only 
begun to solve the problems of assessment that are as- 
sociated with it, persons planning to implement sucn a 
program have no models to pattern and no access to 
instrumentation 'that will permit them to carry out such 
a program even if they are courageous enough to at-, 
tempt it. How else can one account for the fact that 
only a^ handful of programs d9fine teaching competency 
in terms of complex sRill outcomes, and probably no 
more than half a dozen programs in the entire United 
^States define teaching competency in terms of the abil- 
ity to performXthe functions of a certificated teaching 
position or the ability to bring about the outcomes ex- 
pected from the successful execution of a teaching po- 
sition? 

The intricate relationship between competency defi- 
nition and assessment has been recognized for a long 
while! In 1970, in a paper entitled "Jhe I^ocus of Per- 
formance-Based Certification: Knowledge, Teaching Be- 
havior or the Products that Derive fr5m a Teacher's 
Behavior,'' the first author pointed to three levels of 
competency definition that performance-based teacher 
education programs could adopt (Schalock, 1970). These 
were referred to in general terms %s knowledge outcomes, 
skill or "behavioral" outcomes, and product outcomes, 
with product outcomes referring to the outcomes to be 
achieved through the performance of teaching func- 
tions in ongoing school settings. Questions were also 
raised in that paper as to the implications of the yaif- 
ious levels of competency definition for assessment. 
The authors of the elementary models made use of 
similar distinctions in describing alternative foci for 
performance-based teacher education programs (De- 
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. Vault, ct af, 1973), and Turncr^s refinement of this 
three-level distinction to one that involves six levels 
(Turner, 1973) has highlighted the issue even further. 

Up to now, however, little progress has been made 
in developing assessment strategies that parallel the 
three- or six-level distirtctions that have been made in 
^*«Wetcncy definition. Turner's effoits to measure 
teaming skills (1965), the evolution of microteaching 
(Allen and Fortune, 1967), and the recent efforts of 
Pppham in the use of what he terms performance tests 
of teaching proficiency (1967, 1971) provide a begin- 

^ ning to the kind of assessment technology tnat would 
provide for such parallelism, but taken singly or to- 
gether these developments do not as yet provide what 
is needed to implement a performance-based teacher 
education program that defines" teaching coxitpetency in 
its more complex fori^. 

• The purpose of this paf>er is to describe an effort by 
faculty of Oregon CoHege^f Education and the Teach- 
ing Research Division of the Oregon State System of 
Higher Education to develop a system for . assessing 
teaching competency that accommodates the higher 
order definitions of competency. The system thaf is 
being'developcd resg within the context of the elemen- 
tary teacher e^ucat»n program at OCE. and takes as 
its point of departure the following definition: "Teach- 
ing competency is the ability to bring about the out- 
comes expected of an elementary teacher in a certifi- 
cated teaching position.** ^ 

Background 

Defining teaching competency in torms of outcomes 
or product criteria is consistent with the specifications 
of the ComField model for elementary teacher educa- 
tion, one of* the ninci elementary models developed in 
the late 1960\s under the sponsorship of the U.S. 
Office of Education (Schalock, et al, 1968; Schalock,. 
et al, 1970), and with the- specifications set forth in the' 
new process standards for educational personnel devel- 
opment that have been adopted recently by the Oregon 
Board of Education (**Process Standards/* 1973). 
Using the *Trocess Standards" as the document of ref- 
erencCy a teaching competency is defi^ as: 

The demonstrated ability to bring about 
the expected outcomes of a role or func- 
tion included in a job definition; 

and a competent teacher i^ defined as: 

One who has acquired and demonstrated- 
the essential competencies of a profe^ 
sional position and integrates and utili^ 



them effectively in meeting the require- 
ments of that position in accordance with 
its level and certific^ion status. At each 
certification level, thi teacher must also 
provide evidence that he has mastered the 
knowledge and skills assumed to be re- 
quired tor the development of his t&ching 
competence at that level, (p. 18 of the 
April 12 draft of the document)* ^ 

The implications of this set of definitions for the design 
and operation of teacher education programs have 
been spelled out in detail in a paper used by the Board 
of Hducation in the review process given the new 
standards (Schalock, 1973a). 

In the fall of 1972, OCE implemented an experi- 
mental elementary teacher education program that was 
to serve both as a test of the feasibility of the "Oregon 
Process Standards/' and a test of the soundness of the 
principles of the ComField model. It was also to serve 
as a context for research and development, taking as 
its primary objei:tive for the first year of operation the 
development of an assessment system that would meet 
the demands of the most exacting definition of teaching 
competence that is possible (levels 2 and 1 in Turiler^s 
criteria), and do so within the additional constraints 
established "by the ComField model. These include the 
personalization of instruction and assessment, as de- 
fined by Schalock and Garrison (1973), the systemati- 
zation of program operation, as defined by DeVault 
(1973) ancUhe operation of the program within the 
context of a teacher education consortium, as defined 
by the "Oregon Process Standards." 

Forty-three students entered the program. Two full- 
time education faculty, six quarter- to full-time faculty 
from related subject matter arjeas, 43 school supervi- 
sors, and an equivalent of one full-time specialist in 
measurement and evaluation^ staffed the program. The 
program was limited to the measurement and evalua- 
tion'stalfed the program. The program was limited to 
the prestudent teaching aspects of {H^fessional prepara- 
tion, and extended o /er a period of tWo terms (fall and 
winter). Students received 36 hours of college credit 
when they met the requirements of the program. 

A relatively limited sQt o^ developmental goals were 
set for the assessment system during the first year of * 
program operation. The decision \(ds made to concen- 
trate on the development of those aspects of the system 
that would permit the assessment of teaching compe- 
tencies in ongoing school settings at the precertification 
level, moving if time permitted to the development oif 
competency assessment procedures at the level of initial 
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certtfil'ation>* As the year progressed, this turned out 
to mean, operationally, the development of an assess- 
ment system that functioned at two Jevcis of compe- 
tency demonstration: 1) lesson teaching; and 2) short 
term (2 to 5 days), full-responsibility teaching. Lesson 
teaching is the first and simplest context within which 
teaching competency is to be demonstrated in the pro- 
gram. Short-term, full-responsibility teaching is the 
next simplest context for competency demonstration, 
and serves as the staging context for student teaching. 
Short-term teaching can be engaged in only after com- 
petency has been demonstrated in lesson teaching; and • 
student teaching can be engaged only after competency 
has been demonstrated in short-term tea^{B|. 

A third demonstration context received some atten- 
tion during the year, but not as much as the first two 
contexts that have been described. This was a student 
teaching equivalency demonstration context. It required 
full-responsibility teaching for a 5- to 1 0-day period, 
and could be entered only under conditions of excep- 
tional performance in short-term, full-responsibility 
teaching. Successful performance in the student teach- 
ing equivalency context was accepted as evidence of 
the level of teaching competency required to receive 
initial certification. 

As the assessment system rjow stands, it re{^resents 
little more than a beginning of the system that ulti- 
mately must evolve. Two major components of the sys- 
tem have been developed, and a third started, but all 
of these have undergone major revision in preparation 
for the second year of program operation. Undoubt- 
edly^ they will undergo at least on? more major revi- 
sion before they stabilize. !ji addition to the revision of 
what has already been developed, however, the system 
must be extended to cover the assessment of compe- 
tency for purposes of initial, basic, and standard certi- 
fication. This represents a major developmental under- 
taking for as the "Process Standards" ftow reads, initial 
certification requires competency demonstration in a 2- 
to 5-week full-responsibility teaching situation (student 
teaching); basic certification requires competency 
dembnstrattbn in a one- to three-term full-responsibil- 
ity teaching situation (intern or protected first-year 
teaching); and standard certification requires compe- 



'The recently adopted ''Process Staiictards** for educational 
personnel development in Oregon caH for three levels of certi- 
flcation: initial, basic, and standard. Competency demonstra- 
tion is required at all three levels of certification. As level of 
certification progresses the competencies to be demonstrated 
increase i^i number and kind, and performance standards in- 
crease in difficulty. 



tency demonstration in a 2- to 3-year full-responsibillty 
teaching situation after the basic certificate has been 
received. 

Finally, the system must be extended to cover the 
knowledges and skills assumed to be needed !o per* 
form effectively as a teacher. This includes knowledges 
and skills in the various subject matter areas of profes- 
sional education as well as those in the subject matter 
areas to be taught. 

It can be seen from this brief outline that the work 
that remains on the assessment system far exceeds the 
work that has been done. What has been accomplished 
thus far represents only the foundation of the system 
that will be needed in the long run to implement the 
kind of competency-based teacher education program 
that is desired at the college, or that reflects fully the 
specifications of the ComField model or the new **Ore- 
gon Process- Standards." The work that has been done 
represents a beginning, however, for the basic outline 
of the system has been established and several of its 
many com{>6nents have been developed and tested. We 
are at least on the way, and that is more than could be 
said a yeap ago. Because of this, and because so little 
else exists that can be .drawn upon in implementing 
teacher education programs that incorporate higher' 
order definitions of competency, making public what 
has been done thus far at OCE may be of some benefit 
to others. Hopefully it will be of benefit to OCE as 
well, for the response to what has been done may heljp 
clarify problems that are not Seen or have not been an- 
ticipated Qc point to the work of others that could be 
of benefit. ' ' * * " 

Implementing a teacher education program that is 
genuinely representative of the ideals of a model as 
complex as ComField or as demanding* as the **Process ^ 
Standards" -that have been adopted in Oregon Js a' 
monumental task, and any institution that attempts 
such a venture needs all the help it can get. 

B;^re proceeding with the description of the work 
that has been done on^the assessment system thus /ar, 
it is worth noting that the experimental program within 
which the assessment system is being developed was 
judged to be sufficiently successful hi its first year of 
operation that OCE faculty, cooperating school super- 
visors and administrators, and students recommended 
that it be installed as the "regular" program in the ele- 
mentary division of the cdlege. This recommendation 
has been accepted. Operationally this means that be- 
tween 275 and 300 students will move through the 
professional component of the program during the 
coming year, that approximately tho. same number of 



school supervisors and 15 or so college supervisors will 
have to be trained to employ the assessment system that 
•s to be usey in the prograni, and that 20 or more sub- 
ject matter specialists will have to be at least informed 
of the system so that they will be able to relate to it 
meaningfully. The implications of this decision at the 
level of system (Operation are treated in some detail 
elsewhere in the fiaper.-' 

V* 

An Initial View of System Requirements 

When initially planned, the system for assessing 
teaching competence at OCE was designed to accom- 
modate a number of special conditions. These included 
a particular definition of assessment; a particular defi- 
nition of teaching competency; a commitment to the 
principle of gradualism in the demonstration of teach- 
Jng competency; a commitment to the principle that 
measures coming from the system would have utility io 
decisions about instruction, certification, and hiring; a 
commitment to the principle titat measures coming 
from the system would be of sufficient quality that a 
first-rate program of research could be built around 
them; and a commitment to develop an assessmertt sys- 
tem that could be operate^ within the ^constraints of 
the resources available to the college through regular 
funding channels. Since each of these items had major 
impact upon the nature of the system that evolved, 
each Is discusised briefly in the paragraphs that follow. 

i 
I 

The OCE Definition of Assessment 

When planning the experimental teacher education 
program at OCE» the assessment system that was to 
accompany it was to be more than a measurement or 
evaluation system, if by measurement one means the 
assignment of numerals to observations and if by eval- 
uation one means .the assignment of value to numerals. 
It was to incorporate these two sets of operations* but 
include as well the concept of an information manage- 
ment system that serves particular decision-making 
functions. Assessment was seen in the context of the 
OCE prpgram, therefore, as a mechanism ^hat support^ 
decision making. Put in other terms, it was a targeted 
information syjiicm. Two major classes ot decisions 

»A list of the products that have been developed within the 
OCE exp^imental- program^ inciudtog a description of the 
program per se. the teaching competencies pursued within the 
program^ and the system that has been developed for the as- 
sessment of those competencies is available upon tcqiiest. " 
Those interested in obtaining this list, or any of the items re- 
ferred to on it, may do so by contacting the authors. 
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were to be served by the system, instructional decisions 
and program adaptation or design (management and 
pi>licy) decisions, 

Given this concept of assessment, the system was to 
include measures ol teaching competency; pei:fonnanee 
standards for competency demonstration at particular 
levels of certification or prccertifieation experiencg;.- 
specifications as to the decisions to be served by par-/ 
ticular measures of competency; specifications as to the 
structured or mechanisms to be employed in arriving at 
particular classes of decisions; specifications as to the 
form which the data were to assume to facilitate^ each 
particular class of decisions; and an information reduc- 
tion, storage, and* distribution/retrieval system that 
permit:* ihe efficient handling of the data that come 
from the system. The rationale for the system, and a de- 
scription of its ^various pieces and parts, appears in a 
forthcoming book entitled ''Exploring Competency- 
Based Edijcatidh" (Schalock, 1973b). " 

The OCE Definition of Teaching Competeiicy 

The basic defmition ' of teaching competency that 
guided the^ development \of the assessment system at 
OCE has already been cited (see page 59). It Ihas also 
been pointed out that the definiticm of competency 

« adopted by OCE is consistent with the definition pro- 
posed in the ComField model ior elementary teacher 
education, and the definition proposed in the newly 
adopted "Oregon Process Standards'* for the afcredita- 
tion of educational personnel development prc^jams. 
What has not been pointed out is the host^of surplus 
nicanin^ that such a definition carries. 

Perhaps the most troublesome df its various surplus 
meaniQgs is the fact that as defined, competency is al-' 

^ways situation specific. The performance of instruc* 
tibnal planning and preparation functions^ for example, 
ot the performana' of instructional functions W assess- 
ment functions, are always specific td a particular*/ 
group of children in a particular subject matter in a 
particular educational setting, Jhtd require thereby a 
particular set of performance standards! The meaning 
of competence is also always dependent uppn the com^ 
plexity.ol the teaching task to be perfonned. The dem- 
onsrration of teaching competence in the context of les- 
son teaching, for example, has a considerably different 
meaning than the demonstration df teaching compet- « 
ence in the context \)f short-term, full-responsibility or 
intern teaching. Finally, the OCE definition of teaching 
^ competence requires that competence be*' demonstrated 
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in ongoing school settings, and that it meet designated 
performance standards. 

While such meanings may sound complex and 
strange, they arc not at all arbitrary, for they follow 
necessarily from the basic definition of competence as 
the ability to bring about the outcomes expected of a 
certificated Job position. This is so because jobs are sit- 
uation specific. If such meanings are hard to under-' 
stand the reader is referred again to the technical paper 
that spells out in considerable detail the implication oi 
a competency definition of this kind for program 
operation (Schalock, 1973a). 

The OCE De^nition of Gradualism in the 
Demonstra^n of Teaching Coni||>etency 

As indicated previously, the "Oregon Process Stand- 
ards" call for competency demonstration al the level of 
initial certification to take place in a full-responsibility 
teaching situation of no less than 2 weeks duration. 
Th' assunjption underlying this expectation is that a 
demonstration context of any less demanding nature 
would be insufficient as a source of evidence about 
ability to bring about the expected outcomes of teach- 
ing. A year of full-responsibility teaching is suggested 
as the demonstration context for competency assess-, 
ment at the level of basic certification and 2 to 3 years \ 
of full-responsibility teaching, after basic certification 
has been achieved, is suggested as the demonstration 
context for competency assessment for the standard 
certificate. 

The successful performance of teaching functions 
under conditions of full-responsibility for a 2-wcek pe- 
riod is a demanding task. It is also one that is not 
likely to be performed successfully without prior expe- 
rience in teaching. The experimental program^at OCE 
recognized this fact, and was planned to incorporate an 
extensive system of precert'fication teaching in which 
teaching competency could be demonstrated within a 
series of graduated ccmpetency demonstration contexts. 
Two demonstration contexts were to be provided prior 
to the 2-week context in v^hich competence was to be 
demonstrated for purposes of initial certification, les- 
son teaching, and short-term (2-5 d&ysi full-responsi- 
bility teaching. The lesson teaching context was to re- 
quire the preparation and r;esentation of three lessons 
on three separate days and in three different ,subject 
areas. 

Both demonstration contexts were to be provided 
within what amounts to the first two terms of a thrpe- 
terrn profession?! preparation sequence. Obseivation 



and informal lesson teaching were to precede the prep- 
aration and presentation of the three lessons that were 
10 receive formal assessment. Performance standards 
for lesson teaching were to be based upon a summary 
of performance in the three lessons, and were to be 
met before a student was to engage in short-term, full- 
responsibility teaching. The ability to bring about de- 
sired learning outcomes in pupils was not *to be re- 
quired as a competency at the level of lesson teaching, 
though the assessment and display of the learning out- 
comes achieved in lessons were. 

After competence was demonstrated in lesson teach- 
ing, a student was then to demonstrate his competence 
in short-term, full-responsibility teaching. Essentially 
the same competencies demonstrated in lesson teaching 
were to be demonstrated in the new context, but with 
more demanding performance standards. Some addi- 
tional competencies were also to be demonstrated. Per- 
formance standards were to assume muchXthe same 
form as they did in lesson teaching, including not being 
held accountable for brining about desired learning 
outcomes. That was to be a requirement, however, for 
competency demonstration at all levels of certification. 

The demonstration of competence at the level of 
short-term, full-responsibility teaching was to permit a 
student to move on to student teaching, to intern 
teaching, or to a sttident teaching equivalency examina- 
tion. The basic assumption underlying the design of the 
graduated demonstration contexts to be included in the 
program ^s that if students were to perform effectively 
in a 2- to 5-week, full-responsibility teaching atuatfon 
^student teaching) that provides the context for com- 
petency demonstration at the level of initial certifica- 
tion they would have to have a carefully tailored set o^ 
preccrtification teaching experiences that would prepare 
them to do so. 

The Utility of Competency Measures 
For Instruction and ioo Placement 

From its inception ftie assessment system at OCE 
was seen as serving the purposes of instruction, certifi- 
cation, and job placement. Accordingly, in each step in 
its development the system was t^jbe influenced by the 
^ecds of instructional staff, the ^rtifying agency, and 
the personnel officers of school districts. Information 
obtained through the assessment system was to serve 
online instructional (supervision) decisions as well as 
certification and hiring decisions. T^e latter two sets of 
decisions wore to be facilitated by the preparation of 
"competency profiles" that depict competencies demon- 
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strated in each demonstration context. To make the 
preparation of such profiles feasible they were to be 
prepared by eom||>uter. 

The Utility of Competency Measures 
For Research 

One of the major problems that has confronted re- 
search on teacher education, or for that matter re- 
search on the effectiveness of instruction, has been the 
lack of adequate measures on the outcome side of 
teaching. One of the major aims for the OCE assess- 
ment system was the development of a system of meas- 
urement that would eliminate this problem, and Wjy so 
doing enable a first-rate program of research to be es- 
tablished at the college. In order for this to happen a 
long-range, systematically designed program of re- 
search was planned on the quality of the measures 
themselves. The results of the first year of this re- 
search, and the Hh^s which it is to take during the 
coming year, are described in subsequent sections of 
the paper. i • 

The Feasibility of the Assessment System, 
Given the Resources Regularly Availabk 
to the Teacher Education Program at GICE 

While help was to be available fo\ the development 
of the proposed assessment r;ystem through grants from 
Teacher Corps andj the • MtiuuHl Center for the Im- 
provement of Educationo! Systems, U^S. Office of Edu- 
cation, the system was seen as having to ope|*ate within 
the constraints of the resources ordinarily aVailable to 
the teacher education program at OCE once 5t was de- 
veloped. Since the system obviously would require re- 
sources to operate, this qjeani thai ways would have to 
be found to utilize resources presently available in dif- 
ferent ways, or to find as yet untapped resourije.s and 
enlist them in support of system application\ This 
•stance was consciously adopted for two reason. Apirst, 
OCE did riot wish to cjevelop an elaborate assessment 
system and then tind itself in a por;ition of being una- 
ble to operate' it because ft, was too costly. Second if 
the system was to have^ general utility to the field- it 
would have to be functional within the resources ordi- 
narily available to most, teacher education programs. \^ 

The OCE System fox Assessing Teaching \ 
Competency, Year $\ \ 

Starting with the view of system requirements that 
has just been outlined, work was begun on the actual ^ 



developjnent of the system in September 1972. Work 
on the sysicm progressed simultaneously with the dc- 
velDpincnt of the experimental program as a whole, 
ami as with any interdependent effort, steps taken in 
one arena both influenced and were influenced by steps 
taken in the other. Only the results of this developmen- 
tal process will be described in the present paper. The 
process itself is being chror^icled in a monograph that 
describes the development of the experimental program 
at OCE historically (Schalj^ck. Kersh, and Garrison, in 
preparation). / 

/ 

/ 

Teaching Competencies to be Demonstrated 

In order to develop a system for assessing teaching 
competency one has to be clear about the teaching 
competencies to be assessed. Fortunately, the instruc- 
tional staff at OCE were reasonably clear as to the 
competencies that they wished to see demonstrated at 
the lev *l of initial certification, so this step in the de- 
velopmi-ntal pn cess was accomplished with reasonable 
ease. 

It will be recalled that the definition of competency 
adopted at OCE, 'and in Oregon generally, has to do 
with the ability to bring about the outcomes expected 
in the performance of a role or function within a cer- 
tiftcated teaching position. As used in this definition, 
«ole or function refers to the *1argest meaningful classi- 
fication used Ip describing units of work .within a 
teaching position" (p. 19 of the April 12 draft of the 
"Oregon Process Standards"). The teaching functions 
accepted by tHc OCE staff as a point of departure in 
developing the assessment system were as follows: 

• Defining the objectives of instruction ^ 

• Adjusting instruction for the individuals involved . 
(teacher and pupil) 

• Selecting appropriate materials and procedures 
for instruction, given the objectives and individu- 
als involved 

• Organizing the learning environment to support 
instruction 

• Interacting with pupils (for pupil success) in the 
process of inslniction 

• Evaluating studen* growth (cognitive and attitu- 
dinaO 

• Defining next learning steps, and the instruc- 
tional procedures that attend them, given all of 
the above. 

These sev^n functions were proposed itj^itially by Dr. 
Herbert Hitc, now chairn.an of the Department of Ed- 
ucation at Western Washington College, as content for 
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the Com^dd model (Schalock and Hale, 1968). They 
served as thesprimary organizers for instruction and as- 
sessment in tlu* experimental program at OCE during 
its first year of operation.' 

In developing the assessment system around^ these 
seven teaching functions (competencies to be demon- 
strated), the decision was made to obtain evidence as 
to how well the functions were planned as well as how 
well they were carried out in actual teaching. This gave 
rise to two reasonably distinct sets of assessment pro- 
cedures, one focusing upon the assessment of lesson or 
curriculum plans and the othe^ upon lesson or curricu- 
lum presentations. 




H« D. Schalock 

Competency Demonstration Contexts 

* • 
In a program that defines competency in terms of 
the performance of teaching functions in an ongoing 
* school setting the identification of the fcontexts Jn 
which competencies are to be demonstrated becomes as 
critical as the identification of the compctenciesijthem- 
selves. In fact, it has been the OCE experience that it 
is not possible to think in terms of competency without 
thinking in terms of the context in which competency 
was to be demonstrated! Until these ^wo aspects of 

* Teaching functions, of course, are always imtwlded in a 
subject mittter context, and as a consequence thofr assessment 
requires an accompanying assessment of the adequacy of Ihe 
content they carry. 



competency definition were sorted out, little progress 
was able to be made in the development of the assess- 
ment system. 

As indicated previously, the first year of work on 
the system saw the development of assessment capabil- 
ity in two demonstration contexts: lesson teaching and 
short-term (2-5 days), full-responsibility teaching. 
Some work was also done in relation to a student 
teaching equivalency demonstration context, but this 
was not completed. The first two of these demonstra- 
tion contexts deal with precertification competency 
demonstration, and are designed to prepare students 
gradually to demonstrate the level of teaching compe- 
tence required for initial certification. The third is de- 
signed as the context in which competence at the level 
of initial certification may be demonstrated. 

Reasonably detailed specifications surround each of 
the competency demonstration contexts. Those sur- 
rounding lesson teaching illustrate the form and sub- 
stance that they tend to take. 

In partial satisfaction of requirements for completing 
the two-term block sequence thatHBomprises the 1972- 
73 Experimeiital Teacher Education program at OCE, 
each student is to demonstrate competence in teaching 
elementary pupils in appropriate school settings. Each 
ETE student will accomplish this objective by assum- 
ing full teaching responsibility for two to five succes- 
sive school days in an ongoing program of instruction, 
in such fashion that their performance is judged ade- 
quate by the student, the college supervisor, and the 
classitoom supervisor in accordance with the standards 
set for performance m such a situation. 

In preparation for the full-responsibility teaching 
demonstration, each student will prepare and teach 
three lessons, one in each of three different subject 
areas. The preparation and presentation of each lesson 
shall be done under direct supervision and in suth 
fashion that a complete assessment can be made of 
strengths and weaknesses, and that the student may 
profit from the assessment. Satisfactory completion of 
all three lessons is a prerequisite to entry into the full- 
responsibility demonstration. The three teacMng les- 
sons arc to meet the follo^^fing conditio^: 

. 1 . At feast one lesson sJhall involve the teaching of 
-cading, and at least one the teaching of art, 
physical education, or music; 

2. At least one of the lessons shall enable the stu- 
dent to demonstrate ability to teach children 
from diverse cultural or ethnic backgrounds; 

3. At least- one of the lessons shall have as its pri- 
mary or secoitdary ^Instructional objective the 
learning of career awareness; ^ 

4. The desired learning outco nes for each lesson 
should bo limited so that ? lesson normally will 
not require more than sixty (60) minutes of in- 
struction and not less than t .vcnty (20); 



5. Each lesson shall be taught to a grou;^ of stu- 
dents numbering five or more, and at least one 
lesson shall involve an entire classroom or its 
emiivalent; 

6. The student will W evaluated on the basis of a 
specified set of teaching tasks or functions (com- 
plex skills) within each lesson, ranging from the 
planning of the lesson through the assessment of 
pupil learning from the lesson; 

7. Formal lessons may be prepared and taught^at 
any time during the two terms allotted the exper- 
imental program, though the student needs to re- 
member that if all requirements of the program 
are to be met within the two-term period, the 
three teaching lessons will have to be completed 
in time to permit the student to arrange for and 
complete the full-responsibility teaching demon- 
stration; 

8. An objective summary of a student's perform- 
ance in the preparation and presentation of les- 
sons will be provided to the student as soon as 
possible after the completion of either > the plan- 
ning or presentation or each lesson. The student 
will be permitted to offer any reaction or rebut- 
tal to the evaluation which has been made in 
time for it to be of benefit *to all concerned in 
the preparation of the 'n%xt teaching lesson,, or ^ 
the f ull-ffesponsibility demonstration; and 

9. Each student will be permitted to offer any reac- 
tion or rebuttal to the final a§§essment made of 
his performance in the full-responsibility teach- 
ing demonstration before it is finally determined 
whether or not the student has adequately dem- 
onstrated teaching competence in terms of pre- 
viously agreed upon standards. 



Performance Standards for Competency > 
Demonstration 

Another aspect of the meaning of competency that 
had to be unraveled before progress could be made in 
the development of the assessment system was the mat- 
ter of performance standards. This was a particularly 
troublesome concept for it was imbedded in both the 
nature of the competency to be demonstrated and the 
context in which It was to be demonstrated. For exam- 
ple, defining the objectives of instruction was a compe- 
tency to be demonstrated, but there is nothing inherent 
in that competency descriptor that speaks to the qual- 
ity expected (standard) in its performance. It also 
makes no reference to the context in which perforinance 
is to take place. This is equally troublesome since the 
performance standards for defining the objectives of in- 
struction in the context of lesson teaching may be con- 
siderably different than in the context of short-term, 
full-responsibility teaching. Because of this interdepen- 
dency of competency descriptor, the context in which a 
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competency is to be demonstrated, and the perform- 
ance standard set for its demonstration the task of be- 
coming clear as to what the assessment system was to 
do and how it was to do it was more difficult than an- 
ticipated. 

Another level of subtlety andlcomplexlty emerged in 
relation to performance standards as the assessment 
system developed. This was the distinction that had to 
be drawn between performance ratings and perform- 
ance standards. As the system was planned initially it 
was anticipated that performance standards would 
apply to each competency that was being assessed. As 
the system evolved it was discovered that applying the 
concept pf performance standards at that level of detail 
was simpily not functional. Ratings of performance had ^ 
to be apjplied at that level; i.e., at the level of each 
competency descriptor, but it turned out that perfonn- 
ance standards seemed to apply best to performance 
within a particular demonstration context. Thus; as the 
system evolved during the first year of program operas 
tion, pprfon^ance standards came to apply to perform* 
ance patterns across competencies within particular 
demonstration contexts, rather than to individual cofn- 
petency demonstrations. ' ' ' : . 

Such an arrangement in no way lessens the impor- 
tance of individual competency assessments. Thqse^ are 
still the central focus of the system, and their assess- 
ment provides the basis for much of the instruction 
that occurs in field settings. Their assessment also pro** 
vides the basis for arriving at a judgment about per- 
formance standards, for these are defined in terms of 
individual competency assessments within a particular 
demoj;istrafion context. Seen in this way petformance 
standards «serve operationally as tfie exit requirements 
from a particular demonstration context, or entrance ^ 
requirements to another. They also serve as the crite- 
rion "tneasure for certification. The performance stand- 
ards that ^er^e established during the^first year of pro- 
gram opc^ratidn fori, lesson teaching and short-term, 
full-responsibility teaching appear as attachment A. 

The Approach Taken to Measurement 

The approach taken to the measurement of individ- 
ual teaching competencies was one of obtaining care- 
fully delimited professional judgments, in the form of 
rating scale placements, as to the adequacy of a stu- 
dent's performance in a particular demonstration con- 
text. At least two separate professional judgments were 
obtained in relation to each competency demonstration, 
one from a student's college supervisor and one from 
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his schwl supervisor. An evaluative judgment was ixlso 
obtained from a eontent speeialist U a student re- 
quested it. The ratings were designed so as to aeeom- 
modate the impaet of setting differenees on eompeleney 
demonstration. 

The rating scales assumed somewhat different forms 
for evaluating plans to evaluation presentations. In eval- 
uating plans a three-pi^int scale was used; in evaluating 
presentations, a five-pi>int scale. This difference oc- 
curred because ratings were applied only to items in a 
plan that were acceptable; i.e., if an item was unac- 
ceptable it had to be modified until it was. As a conse- 
quence the ratings applied to the plan were really only 
the upper three scale positions of the fivc-piMnt scale 
applied to teaching presentations. 

In addition to' a judgment as to individual competen- 
cies^ raters' were asked to provide an overall judgment 
as to the quality of a plan or a presentation. These lat- 
ter judgments were to be made only after all indiviciual 
competeocies had been assessed. To illustrate the na- 
ture of the overall rating system the forms used in the 
sfpplieation of the scales to lesson teaching are ap- 
pended as attachment B. 

Two other features of the rating systen>> are worthy 
of note, namely, the listing of items under .each compe- 
tency descriptor that give focus to the descriptor and 
the requirement that the behavioral or product indica- 
tors relied upon by a rater in responding to an item be 
recorded. In the evaluation of plans, each of the focus- 
ing items was rated (which turned out to be a poor 
practice). In the evaluation of presentations each fo- 
cusing item was attended to, but only the competency 
was rated. This latter strategy proved to be reasonably 
tunctibnal, and provided the basis for an important re- 
vision pf the forms for the coming year. 

The' decision to ask raters to record the indicators 
they relied upon in making a. particular judgment grew 
out of the need to make the ratings as objective as 
possible. It was felt that the inclusion of the request to 
record indicator statements would help bring about ob- 
jectivity in two ways. First, it would force raters to at 
least think about, and hopefully identify, the indicators 
used. Second, by compiling a listing of the indicators 
used by different raters in different settings, a guide on 
indicator use could be prepared that could be of value 
in training raters. Unfortunately, not enough care was 
directed to this aspect of the rating process during the 
first ye<ir of program operation, and a relatively spot- 
ted record of indicator use was the result. In the coming 
year the rating methodology has been changed in this 



regard, however, and indicator usage assumes a more 
prominent place within it. 

Data Management and Utilization 

Only the most rudimentary system was developed 
during the first year of program operation for the inan- 
agement and utilisation of the data coming from the 
competency assessment system.'* A work-study student 
was responsible for the distribution of the various eval- 
uation forms to students and college supervisors. Stu- 
dents were then responsible for getting the forms they 
received to their school supervisor, and for returning 
the completed school supervisor's forms to the college 
supervisor. A record of completed forms was main- 
tained in each college supervisor's office for'each of the 
students they were sponsoring, and a data summary 
sheet was prepared from each set of forms for use in a 
permanent file. The data recorded on these summary 
sheets were then put In com|)uter storage for purposes 
of competency profile preparation and research^ For 
illustrative purposes, the summary data form, for lesson 
teaching appears as figure I. The translation of this 
information to the computer permitted a series of meth- 
odological studies to be undertaken oti the adequacy of 
the measures coming from the syktem (see the next 
section of the papeH, ^nd preliminary work was able 
to be done in the preparation of competency profiles. 
.I his task was sufficiently complex, however, that its 
completion was not possible during the first year of 
program operation. * 

Data management procedures that served dnline de- 
cision making in relation to lesson and short-term, 
t'uil-responsibility teaching were Tcft largely to th^inge- 
nuity of the college and school supervisors involved 
Several guidelines to data use were provided, hoWever. 
First, all conferencing with students about plans or 
performance ip the classroon^ was to be data based, at 
least at point of departure. Second, a student had the 
right and obligation to quarrel with ratings or indicator 
statements whenever he or she felt they were inaccu-, 
rate or unfair. Third, discussion of performance in thej 

i 

•Two assessment systems were actually developed iq sup-, 
port of the experimental program at OCE during Its first year 
of operation. The first is the wompetency assessment system 
ihat is d:scribed in the present paper. The second is a pro- 
gram asscssm.:nt system that is designed to systematically col- 
lect data on all aspects of program operation and make that 
data available at a time and in a form that facilitates program 
adaptation decisions. The program adaptation system is de- 
signed on the same principles as the competency assessment 
Nystcin (see pp. K to 14) and should therefore have the same 
degree o? transportability. 
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classroom was to occur as close as possible in time to 
actual performaiice. Fourth, the college and school su- 
pervisor would confer as to the overall adequacy of the 
performance of a student before a student was permit- 
ted to advance to the next level of competency demon- 
stration, even though the college supervisor was the 
person to make that judgment ultimately. 

Within these broad guidelines all possible combina- 
tions of procedures and schedules were followed. No 
reduction or synthesis of the data on the evaluation 
forms occurred before or after discussions with stu- 
dents, except for the summary that was prepared for 
permanent file and computer use. As the reader will 
see in the next section of the paper the failure to direct 
greater attention to the management and utilization of 
the dafa coming from the assessment of lesson and 
short*term teaching had its consequences. 

Data management procedures were more exacting in 
relation to the student teaching equivalency demonstra- 
tion, even though much less work had been done de- 
velopmentally on that demonstration context than the 
others. This undoubtedly reflected the fact that certifi- 
cation decisions were involved. The procedure that was 
developed to handle this level of decision required that 
a jury of college faculty and classroom teachers who 
had not served as the student's supervisors examine all 
data available on a student's fjerformance and arrive at 
a decision concerning the match between performance 
and performance standards. The student's sponsor pre- 
sents the data called for in this regard and provides the 
necessary context building that permits the jury to view 
the data in perspective. A segment of video tape show- 
ing the student teaching is iticluded as a part of the 
student record. 

This procedure was tested on five occasions during 
the first year of the program and found to be reasona- 
bly satisfactory. It needs to be pointed out, however, 
that firm performance standards were not operating on 
those occasions and so our experience with these pro- 
cedures did not represent a full test of them. 

Data oh System Operation 

In order to determine how well the assessment sys- 
tem was working in the context of the experimental pro- 
gram, analyses were made of the various student 
applications of the system, the completeness of infor- 
mation coming from the system, the trustworthiness of 
that information, etc. Studies of this nature were car- 
ried out midway through th^ program and then again 
at its completion. The data reported in the paragraphs 
that follow arc based on these analyses. 



Of the 43 students that enrolled formally in the ex- 
perimental program, 42 attempted lesson teaching and 
38 of these met the performance standards set for that 
demonstration context. All 38 of these students then 
attempted short-term, full-responsibility teaching, and 
34 of them met the performance standards set for that 
context. On the basis of these figures approximately 
three-fourths of the students who entered the program 
met competency demonstration requirements for exit 
from it, and thereby met entry requirements for stu- 
dent teaching. Five of the 34 students, however, chal- 
lenged student teaching through the student teaching 
equivalency demonstration, and three of those five 
were judged competent at the level of initial certifica- 
tion. 

Taken at face value these data would suggest that 
the assessment system was working well. In some re- 
spects tharis a fair judgment. A larger proportion of 
students dropped or were dropped from the experimen- 
tal program than is typically the case in the nonexperi- 
mental form of the program. College and school super- 
visory staff also reported that the evaluations that they 
were able to make of student's performance were 
sharper and more detailed than they had ever been 
able to make before. Staff also reported that the data 
base provided by the ratings greatly facilitated instruc- 
tional activities that accompanied the supervisory proc- 
ess. 

Two other > s of data, however, force caution in in- 
terpreting how well the system worked. The first is a 
set of data that has to do with the conscientiousness of 
performance rating and documentation by various eval- 
uators. These data are summarized in tables 1 and 2, 
and are informative on a number of counts. First, it is 
immediately clear that the rating forms were not ap- 
plied in all cases, and often times when applied they 
were not attended to completely. Second plans are 
rated more consistently and more compi'itely than 
presentations. Third, the conscientiousness of rating 
and documentation varied by class of rater, with the 
school supervisors generally being the most conscien- 
tious about filling out the forms completely. Finally, 
very few content specialists from the college faculty ap- 
plied the forms to either plans or presentations, and 
when they did they were not overly conscientious about 
their use. 

If taken at face value these data would suggest that 
the assessment system .was essentially nonfunctional.^ 
This would be an over-interpretation, howevcf* for 
while the application of the system obviously left much 
to be desired the data that appear in tables 1 and 2 
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Table 1. Coitipleteness of KatiiiKs on Competency in 
Lesson Planning and Presentation 





C(?tlef(e 


School 


Cont 


Elements Rated 

- « - 


Supervisor 


Supervisor 


Specii 


Plans 








all elements rated 




Lesson 1 


20 


30 


3 


Lesson 2 


20 


23 


7 


Lesson 3 


17 


27 


4 


some elements rated 








Lesson I 


•t 


3 


i 


Lesson 2 


1 


6 


2 


Lesson 3 


1 


2 


0 


no elements rated 








Lesson 1 




3 


1 


Lesson 2 


0 


1 


2 


Lesson 3 


0 


I 


0 


Presentations 








all eiemenis raieu 








Lesson ! 


16 


20 


1 


Lesson 2 


7 


19 


2 


Lehson 3 


3 


17 


0 


some eien7ents rated 




r 




Lesson 1 


3 


4 


0 


Lesson 2 • 


1 


4 


1 


Lesson 3 


I 


3 


0 


no elements rated 








Lesson 1 


11 


10 


4 


Lesson 2 


12 


8 


8 


Lesson 3 


15 


9 




4 



Table 2. Completeness of Ratings for Sl^-Term, 
Full-Responsibility — ^ 





College 


School 


Content 


Elements Rated 


Supervisor 


Supervisor 


Specialist 


Ptaus 


(N = 24) 


(N = 29) 


(N=l; 


all elements rated 


18 


28 


1 


some elements rated 


4 


1 




no elements rated 


2 






Presentations 


(N-24J 


(N = 28) 


(N*l) 


all elements rated 


1 


16 


I 


some elements rated 


3 






no elements rated 


20 


4 





have a number of explanations. First* the system was 
instigated with essentially no staff preparation. Second, 
some items within the system were badly in need of re- 
vision « and as a consequence many c valuators simply 
chose to omit them and deal only with those that made 
sense or were able to be managed. Third, the supcrvi- 
Si>ry load on the coUcgc staff became so heavy nea. the 
end of the program that they were cs.sentially unable to 
meet the demands that were placed upon them. This is 
reflected in the high proportion of ratings missing 
the second and third lesson presentations, and in the 
short-term* full-responsibility teaching situation. Fi- 
nally* no expectations were established nor held in the 



prograhi for content specialists to apply the system to 
either plans or presentations. Students were free to ask 
their participation in the rating process if desiiud; or 
content specialists could ask to become involved in as* 
scssing a plan or the performance of a particular stu- 
dent« but this was not a matter that received a great 
deal of attention in the program. 

In some respects, then, given the circumstances that 
surrounded the development and application of the sys- 
tem, it is possible to be delighted with the extent of the 
systcm*s application iind the conscientiousness with 
which it was applied. The data arc particularly encour- 
aging in this regard for school supervisors. 

Three additional sets of data support a sense of 
hopefulness about the systeno and its operation in the 
context of the experimental program. The first deals 
with a set of analyses that were, carried out to deter- 
mine the sensitivity of the ratings. Two kinds of sensi- 
tivity indicators were .used, the extent to which compe- 
tency performance measures varied for an individual 
student, and the extent to which performance profiijbs 
varied across students. The assumption underlying both 
analyses was that in most cascis variability should be^ 
found in individual competency demonstrations within 
the profile of any single student, and that there should 
be variations in competency profiles across students* It 
was further assumed that if such variability were ob- 
served this could be taken as evidence of the sensitivity 
of the measures. 

The results of these analyses were in the direction 
desired. While some students were found to vary rela- 
tively little in the competencies demonstrated, most 
students varied considerably. More ^importantly, they 
tended to vary in all possible ways. For example, some 
students were consistently high across performance 
measures, some consistently low, and some both high 
and low. Similar variability was found between stu- 
dents. 

The secopd set of data that arc encouraging of the 
system*s potential deals with the extent of agreement 
on ratings of student performance between independent 
raters. A number of analyses of this kind were made, 
though obviously they were limited by the incomplete- 
ness of the data as reflected in tables 1 and 2. Never- 
theless, by the close of the first term of the program 
22 lesson plans and 12 lesson presentations were found 
that were sufliciently complete to permit interrater 
agreements to be calculated. On the basis of these 
calculations, level of agreement was approximately 80 
percent for the items rated in lesson plans and 75 per- 
cent for the items rated on lesson presentations. No 
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interratcr agreements were calculated for short-term 
teaching. 

I he third set of data that arc encouraging deals with 
rating patterns of evaluators. In these analyses, cvalua- 
tor ratings across students were the basis for compari- 
son, and were analyzed independently of the students 
on which the ratings were made. Pattern analyses were 
run that compared a) college and school supervisors 
ratings, b) one college evaluator's ratings with the rat- 
ings of another, c) ratings provided by school supervi- 
sors in one school with those of another, and d) rat- 
ings provided by content specialists with those 
provided by both college and school supervisors By 
and large these analyses showed that while there was 
some tendency on the part of all evaluators to skew 
the ratings toward the upper scale values, and some 
tendency ft^r rating patterns to reflect the individuals 
doing the rating (for example, one college supervisor 
will tend to rate biuher or with less variability than an- 
other), overall rating patterns tend to be roughly 
equivalent. This is especially the case as ratings being 
compared increase in their generality or larger numbers 
of ratings are compared. 

* l*his last point Is illustrated by the histograms pre- 
sented as figures 2 and 3. Figure 2 shows the rating 
patterns of two college supervisors for competence in 
classrix>m management in rhe context of lesson teach- 
ing. Figure 3 shows the rating patterns of xill college 
supervisors and all ^hool supervisors for the same 
measure. Even though one would expect greater simi- 
larity between the two college supervisors than between 
college and sch(H>l supervisors, this was not the case. 
The greater similarity in rating patterns reflected in 
figure 3 can best be accounted for by the effect of large 
numbers entering the picture. 

There tended to be less variability in rating patterns 
around plans on the part of all evaluators than around 
presentations. 

As indicated throughout this discussion, the data on 
system performance are both encouraging and discour- 
aging. On the encouraging side there is evidence that 
the measures provide reasonably sensitive discrimina- 
tions in relation to pupil performance, and that raters 
tend to provide reasonably similar ratings when observ-' 
ing the same students and reasonably similar patterns 
of ratings when observing across students. Add in this 
the opinions of both college and .school supervisors 
that the data that come from the system help them 
make better judgments about competence and provide 
better instructional help than has heretofore been pos- 
sible, there is reason to be hopeful about the potential 



of the system. It should also be noted on the hopeful 
side that both college and school supervisors indicated 
strong support for the continued use and further devel- 
opment of the assessment system, and that its use is 
not impossibly expensive. The resources invested in the 
development, opcratii.n, and adaptation of the experi- 
mental program as a whole 'this past year are summa- 
rized in figure 4. Costs most directly attributable to the 
assessement function within the program are those as- 
sociated with increased school supervisory time, 
development costs, adaptation costs, and reseandi 
costs. 

All things considered, these are reasonably encour- 
aging data. On the discouraging side, however, there is 
evidence that if the system is to function effectively, 
and if the mciisures are to be of a quality that permits 
a great deal of confidence to be placed in them, there 
is still much to be done. Major revisions within the 
system itself must be made and an effective procedure 
be devised to assure care in its application. It is to the 
proposed modifications in the system for the coming 
year that we now turn. 

An Expanded View of System Requirements 

On the basis of the data just reviewed, and recom- 
mendations from college and school supervisors, stu- 
dents, and assessment personnel, major changes are 
being made in the competency assessment system for 
the second year of program operation. These changes 
will be reviewed in some detail since they represent 
what appears to be m«ijor advances in the methodol- 
ogy. Jo provide continuity with the description of the 
assessment system provided previously, the headings 
used in that description will be used again. 

Teaching Competencies To Be Demonstrated 

It will be recalled that seven teaching functions or 
competencies constituted the core of the experimental 
program during the first year of operation (see p. 64). 
For the coming year the list has been expanded and 
organized into clusters of competencies. As the list 
presently stands, four competency clusters are to be 
demonstrated in the context of lesson teaching: Plan- 
ning and Preparing for Instruction; Performing Instruc- 
tional Functions; Performing Assessment Functions; 
and Displaying Pupil Achievement. A fifth cluster of 
competencies has been idded to these four basic clus- 
ters for demonstration in the short-term, full-responsi- 
bility teaching context and in student teaching. This is 
a set of competenL^es that has to do with Interpersonal 
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RESOURCES UTILIZED IN THE FIRST YEAR OF THE ETE PROGRAM 

(44 students enrolled in the program, 37 completed it) 



Fall 



Regular 
'^Junior-Block'' 
Progrmn 

Winter 



Spring 



Fall 



Experimental 
Vunior^Block'' 
Program 

Winter 



Spring 



Program Operation 
Education Faculty 
Subject Matter 
School Supervisors 
• 

Secretary/Clerical 
Supplies/Services 
Administration 

Program Development 
Education Faculty 
Assessment Faculty 
Secretary/Clerical 
Supplies/Services 
Administration 

Program Adaptation 
Education Faculty 
School Supervisors 
Students 

Assessment Faculty 

Secretary/Clerical 

Supplies/Services 

Program Related Re- 
search 



1.5 PTE 
1.5 FTE 
2hrs. . 

p/wk. 
Normal 



1.5 FTE 
1 . 25 FTF 
2hrs. 

p/wk. 
Normal 



1.5 FTE 
1.75 FTE 
3 hrs. 
p/wk. 

+50^;, 

Normal 
+ 10% 



.5 FTE 
1,0 FTE 

•75 FTE 
$750 
+ 10% 



$500 



1.5 FTE 
1.50 FTE 
5.25.hrs, 
p/wk, 
+50% 
Normal 
+ 107o 



,5 FTE 
l.GFTE ^ 

.75 FTE 
$750 
+ 10% 



$750 



.50 FTE 
6 @ 5 hrs. o/wk. 
12 @ 5 hrs, p/wk, 
1,0 FTE 
1.0 FTE 
$500 



$500 



Figure 4. Resources utilized in the first year of the experimental program at OCE (43 students enrolled in the program, 

34 completed it) 



Interaction. The separate competencies to be demon- 
strated within these five clusters are as follows: 

Competency Cluster L 

Planning and Preparing for Instruction 

• Definmg learning objectives and the indicators of 
their achievement 

• Planning instructional activities, materials, and 
procedures 

• Planning for the assessment of learning 

Competency Cluster II* 
Performing Instructional Functions 

• Conveying the objectives of instruction 

• Adapting instruction to context 

• Managing the instructional process 

• Managing unexpected events 



Competency Cluster HI. 
Performli^ Assessment Functions 

• Assessing learning before instruction 

• Assessing learning dunng instruction 

• Assessing learning after instruction 

Competency Cluster IV. 
Displaying PupO Achievement 

• Displaying prelesson and pc^tlesson achievement 

• Displaying learning gains that result from in* 
struction 

Competency Cluster V. 

Enhancing Interpersonal Relationships 

• Acting responsibly in terms of the feelings, 
needs, and wishes of others 

• Working constructively in task-oriented situa- 
tions with others 



It is expected that this list of competencies will con- 
tinue to expand or be refined as the assessment system 
is applied and test«:d in student teaching and intern 
contexts^ or in contexts designed to give evidence of 
competency demonstration for purposes of standard 
certification. 

(competency Demonstration Contexts 

^o changes are planned for the demonstration con- 
texts that were established in the first year of program 
operation. In fact» with the formal acceptance of the 
"Process Standards" for the preparation of education 
personnel by the Oregon Board of Education, the dem- 
onstration contexts thus far established in the program 
appcar^tiv be more appropriate than initially antici- 
pated. The competency demonstration contexts that are 
now pri>posed for the program, and their relationship 
to level c\f certification, is as follows: 

Contexts tor Competency Demonstration 
Prior to Certification 

• Lesson teaching 

• Short-term (2 to 5 days), full-responsibility 
teachitig 

Contexts for Competency Demonstration 
' For Purposes of Certification 

e Initial certification: student teaching (2 to S 
we€^s), M[>r student teaching equivalency demon- 
stration to 10 days) 

• Basic certification: intern teaching < 10-30 
weeks) or. pfotected first-year teaching 

• Standard certification: 2 to 3 ye3rs of full-time 
teaching after the basic certificate has been re- 
ceived. 

Performance Standards for Competency 
Demonstration \ 

It will be recalled that performance standards were 
established during the first year of the program for 
competency demon^itration in lesson teaching and in 
short-term, full-respi)nsibility teaching only. It will also 
be recalled that these standards received very little for- 
mal testing. As a consequence* with one important 
change, the second year of the program is being en- 
tered with thfi same jx^rformancc standards for lesson 
and short-term leaching that were set in the first year 
of the program » The change is to fequire a summative 
competency assessment for exit from the preccrtifica- 
tion phase of the program. The assessement procedure 
proposed is much like that outlined* for assessment at 
the level of initial certification (exit from student 
teaching), for it is also to involve aj^jury of indepcnd- 
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cnt judges. These judges 'are to consist of college and 
school faculty that did not carry supervisory responsl- 
* bility for a student and they are to decide whether 
demonstrated performance in the short-term, full-re- 
sponsibility teaching situation meets the performance 
standards that have been set for it. A segment of video 
tape ^showing the student teaching is also to be in«* 
eluded as part of the student's record. 

One of three decisions will be reached as a cocse** 
qucnce of this summary assessment: 

• the student has passed the teaching competency 
requirements at the precertification levels and may 
enroll in student teaching or internship 

• the student is eligible for student teaching or in- 
ternship, but may enroll only after completing 
specified tasks or submitting additional evidence 
of teaching competence 

• the student is not eligible for student teaching or 
internship, and must remain in^ the prestudent 
teaching part of the program until evidence can 
be provided that all teaching, competency require- 
ments for entry into student or intern teaching 

. have been met. 

Specific criteria have not as yet been established as a 
basis for any one of these decisions. 

It is anticipated that with use, the performance 
standards that, have been set for lesson teaching and 
short-term, full-responsibility teaching will be sharp- 
ened or in some other way modified. It is also expected 
that the jury system that is being proposed for sum- 
mary judgment of competence in short-term teaching 
will be applied to all certification judgments within the 
program. Finally, it is expected that all certification de- 
cisions will rest heavily on evidence as to the ability of 
a prospective teacher to bring about the learning out- 
comes expected for pupils. Almost no thinking has 
been done within the program about the nature of 
• competency demonstration requirements for purposes 
of certification beyond those that have been mentioned. 

The Approach Taken to Measurement 

Major refinements have been made in the approach 
. \^ still one that involves ratings. Seven major changes 
have been made in the rating system: 

• all ratings are made in terms of a five-point scale 
(during the first year teaching plans were rated 
on a three-point scale while teaching performancfe 
was rated on a five-point scale i 

• ratings are provided only for competencies and 
competency clusters (ratings were required during 



f 

the first year of the program for the items that 
elaborated a competency as well) 

• all ratings are to be recorded by entering a nu- 
merical score in a box opposite the competency or 
competency cluster to be rated, rather than mark- 
ing a position on a continuous scale (it is antici- 
pated that this priKcdure will force more care or 
attention to be given each individual judgment 
and its recording) 

• the rating scale positions are more carefully 
anchored in the attributes that define each scale 
position 

• examples of the Indicators that can be relied 
upon in arriving at a particular rating scale judg- 
ment have been provided on the rating form 

• the rating forms have been revised so that they 
invite easier recording of the indicators used in 
arriving at a particular judgment 

• all competency statements, and the items that 
are intended to defme them, have been edited and 
field tested for their clarity and meaning. 

In combination these changes are designed to make 
the rating process both more manageable and discrimi- 
nating. 

The forms as a . whole have not as yet been field 
tested, but initial reactions to them by college and 
school supervisory staff have been most encouraging. 
To illustrate the content and format of the new assess- 
ment forms» sample pages from the assessment battery 
for lesson teaching are appended as attachment C. 
Complete copies of the lorms can be obtained upon 
writing the authors. 

One further change has been made in the assessment 
forms that is of major consequence. Tliis is their organ^ 
ization by the source of indicators relied upon in 
making judgments about the competencies being rated. 
Accordingly* Planning and Preparation Functions, and 
Achievement D» May Functions, are judged in terms of 
products of a tw. .her*s behavior; Performance of** As- 
sessment Functions arc judged on the basis of a teach- 
er's behav^^)r per se: and Performance of Instructional 
Functions and the Enhancing of Interpersonal Rela- 
tions are judged both on the basis ot teacher behavior 
and pupil behavior. These distinctions will be noted in 
both the directions given to evaluatofs and in the sam- 
ple indicators. 

Data Management and Utilization 

As yet (July 1973) the specific data management 
and utilization procedur&s to be implemented during 
the second year of the program have not been estab- 
lished, though a set of quality assurance procedures 



have been agreed to and an extensive program of re- 
search on the quality of the measures coming from the 
assessment system is being prepared. Specific proce- 
dures for the distribution and collection of forms, for / 
the utilization of the information contained on the / 
fofms for instructional purposes, and for the utilization / . 
of that information for decision purposes relative to^* 
movement from one demonstration context to the nexlf, 
are still to be defined. f 

Faking steps to insure the quality of the nleiswres 
obtained through the assessment system is particularly 
critical in the coming year because of expandea use. 
Fifteen college supervisors, up to three hundred stu- ' 
dents and school supervisors, and a dozen or so con- 
tent specialists will be applying the system throughout 
a half dozen school districts. Well-defined quality as- 
surance procedures must be implemented' if the data 
xroming from the various users of the system are to be 
at all trustworthy. 

Two strategies make up the quality assurance plan. 
First, it calls for a careful inservice education program 
to be provided on system usage. Second, it calls for 
systematic checks on the quality of ratings being made. 
These checks will be made midway through each term 
that the program is offered, and at the end of each 
term, inservice programs will be designed on the basis 
of the information obtained through these checks, and 
data management procedures will be elaborated as 
needed. The research that is planned on the quality of 
the measures follows the same general lines as the re- 

. search pursued in the first year of the program^ though 
it will be extended in quality and scope. One addition 
will be the systematic study of indicator usage. The 
computer programs needed to carry out such research 
have been developed and tested sd the results of these 

\studies will be able to be acted upon as the program 
|>rogresses. 

The OCE Assessment System in Pecspective 

To those who have managed to work their way 
through the paper it must be abundantly. clear that the 
assessment system being developed at OCE is a long 
way from completion. Thp parts of the system that 
have been developed will obviously undergo further re- 
finement and the more complex parts of the system are 
yet to be developed. Problems of behavior and product 
' sampling within demonstration contexts, performance 
standards ^r^ ^re demanding demonstration contexts, 
and the development of measures of competence that 
• are trustworthy are still to be confronted. As planned 
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now, the completion of the total s>stem in a lorm that 
^will periiiit its use with known confidenee is targeted for 
the close of the 1975-76 acaileniic year. 

Whilir much remains to be done on the 5>>stenu and 
major problems are yet to be' resolved, a gcx)d deal l)f 
progress has been made. 'I'he b^>sie outline of the s>v 
teni is complete; the basic constructs, dimensions, and 
methodologies of the system have been defined and im- 
plemented;*data management a<ncf utiii/ation strategies, 
thbugh primitive, have been established; quality assur- 
ance mechanisms have been developed and tested; ai?d 
the system in its broad outline has been found to be 
both acceptable and useful to college faculties, school 
supervisors, and students. The system has also been 
» found to be manageable in terms of cost, particularly 
when developmental costs are differentiaied from sys- 
tem operations costs. While there is obviously much 
that remain}l to be done, the rudiments of the system 
have at lehst been developed and tested. In the judg-^ 
m^nt of the authors this in itself represents a reasonable 
^gain for the world of education. 



The assessment effort at O^E needs to be viewed 
within still another context,* however, and that is the 
context Qi" research on ^jdueation and teacher education. 
One of the great handicaps of research on the effec* 
tiveness of teachers has been the lack of strong meas* 

' ures of effcetivehess. Without such measures, np matter 
h(Sw g? ! id the design or elaborate the analysis, signifi** 

) cant leiationshtps are not likely to be found. A weak 
dependent or/criterion measure will defeat a strong re- 
search design and arialysis everytime. 

In the authors* judgment the work*that has been ini- 
tiated at OCE in the'^jea of ^.competency assessment 

• Represents a * n^Jor step toward the resolution of the 
crilarion* problem in teacher effectiveness research. If 
' ail goes according to plan, 3 years from now the work 
begun last year will be completed, and for Ihe first 

^ .time a measurement system may be availa1)le that will 
meet the demands of research that can make a ^iffcv'^ 
ence. When that time comes there can be a hopefulness 
/d\>o\\t educational research thai 'has been massing for a 
Long while. ^ 
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Attachment Performai\ce Standards 



PERFORMANCE STANDARDS 
FOR LESSON TEACHING 

The first, and simplest context within which teaching 
competency is to be demonstrated in the ETE program 
at OCE is lesson teaching. Competency at this level 
of teaching must be demonstrated before a student is 
free to engage in full-responsibility teaching. At least 
three Ipssons must 4>e taught for purposes of formal 
evaluation, and standards must be met for both their 
preparation and their presentation. 



Standards for Lesson Planning 

In preparing for lesson teaching a reasonably detailed 
lesson plan must be prepared and reviewed by both 
college and school supervisors. The standards set^ for 
the preparation of plans are item specific standards, 
that is, both the c6llege and school supervisor must indi* 
cate, independently, that every ife<n to be attended to 
in the plan ha^^ been dealt |with satisfactorily. This 
standard mwt be met before; the lesson can be pre- 
sented to children. If a plan doips not meet this standard 
upon its initial review, it must be revised until it does. 



Standards for Lesson Presentation 

The standards set for performance in the presentation 
of lessons are pattern standards, that is, they apply to 
the pattern of performance demonstrated in the presen- 
tation of three or more lessons. Two standards are to 
be applied to^ the performance record of a student on 
the three or more lessons presented: 

— evidence of favorable performance on each of the 
teaching functions assessed in at least one of the 
three lessons presented; , 

— evidence of favorable performance on the prepon- 
derance of teaching functions assessed in the three 
lessons presented. Preponderance is defined here 
to mean at least 75 {K*rcent of the functions as- 
sessed in the course of the three lessonii^^scnted 
will reflect evidence of favorable perfoN^nce* 
and no more than 25 percent of the functions\as- 
sessed will reflect evidence of unfavorable per- . 
formance. 



\ 

PERFORl^ANCE STANDARDS 
FOR SHORT-TERM (2-5 DAYS) 
FLjLL-RESPONSIBILITY TEACHING 

^hen siuden^ in the ETE program meet perform- 
ance requiremerks in lesson teaching they are free to 
enter the first fiill-responsibiiity teaching Experience 
tha^ is provided \in the program. This is what is 
termed a short-terYn, full-responsibility teaching con- 
text. This experien\:e requires a student in the pro- 
gram to assume futl responsibility for planning and 
carrying out instructbn in the schools for a minimum 
of 2 days and a maximum of 5. 

. Three kinds of statifiards are applied to a student's 
performance in short-t0rm, full-responsibility teaching. 
Two of these correspoipd to the standards applied to 
lesson teaching! The thifd set of standards pertains to 
the utilization and management of affect. 

Standards for Curriculum Planning 

As in the case of individual lesson teaching, a cur- 
riculum plan for short-tern;i, ful^responsibility teaching 
must be approved before {Reaching can be undertaken. 
This requires that a reaspnably detailed curriculum 
plan be prepared for a 2- to 5-day demonstration (as 
used here a curriculum plan consists of a number of 
individual lesson plans, and the relationships if any 
between them), the curriculum plan be reviewed by 
both college and school supervisors by lesson plan, and 
that both the^ college and sohool supervisors must in- 
dicate, independently, that e^ch of the items to be 
attended to in the plan as a whole has been dealt 
with satisfactorily. In keeping with the generally more 
demanding requirements of i the short-term, full-re- 
sponsibility demonstration aU, lessons to be presented 
within the 2- to 5-day teachiing period must n^et ac- 
ceptable standards before teac|iing can begin. 

Standards for Curriculum Presentation 

Standards for curriculum presentation in short-term, 
full-responsibility teaching are more demanding, and 
cover more aspects of teaching, than do the standards • 
for individual lesson presentation. The first standard 
assumes the same form as one of the standards set for 
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the presentation of individual lessons. This is the pat- 
tern standard that requires evidence of favorable per- 
formance on the preponderance of teaching functions 
assessed in the sum of the lessons presented in. the 
2 to 5 days of full-responsibility teaching. The second 
standard for lesson presentation is also a pattern stand- 
ard, and requires that over the course of the lessons 
presented in the period of full-resptmsibility teaching: 

— variety in learning activities will be proyided; 
— variety in cognitive functions and levels in pupils 

will be exercised; 
— variety in affective expressions will be employed 

in teaching; and \ 
— positive feelings in pupils, such as excitement and 

interest, vyill be utilized in their learning. 

Standards for Affect Management 

The standards set for the management of affect in 
short-term, full-responsibility teaching take as their 
focus four dimensions of affective expression: 



— teacher responses to instances of pupil affect; 
— the management of pupil responses to instances of 
pupil affect; 

— the anticipation of pupil upsets and disruptions, 

and their redirection; and 
— the management of pupil upsets and disruptions 

when such occur. 

Performance standards in relation to these dimen- 
sions of affective expression require that during the 
course of the full-responsibility teaching experience a 
student need only to perform effeptively three of the 
four dimensions specified (any three will do), and 
that he or she needs to perform to this level on only 
one of the 2 or more days that he engages in full-time 
teaching. Such a. standard reflects^ the view that the 
management of affect fn a classroom is a complex 
matter, and that in an initial full-responsibility teaching 
situation, performance standards set for it should not 
be particularly demanding. 
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Attachment B 

DEMONSTRATION CONTEXT: LESSON PREPARATION AND PRESENTATION 



LESSON PLAN 

{Attach to your Lesson Blan and Lesson Plan Evaluation Form) 



Student's Natne 



Lesson Number 



RFCORD OF NEGOTIATION 



APPROVAL TO NEGOTIATE THE LESSON 
WITH A SCHOOL SUPERVISOR 



APPROVAL TO PREPARE A FORMAL PLAN 
FOR THE LESSON 



College Supervisor 



STUDENTS TO BE TAUGHT 

School— 

Grade 

Number 

Special Characteristic:- . - - 



Date School Supervisor Date 

4 * * 

CONTEXT DESCRIPTION 

eONTENT TO BE TAUGHT 

Area-- - - 

Expected Learning Outcomes ^ 



DATE(S) OF LESSON PRESENTATION 



TIME(S) OF LESSON PRESENTATION 



SUMMARY EVALUATION OF LESSON PLAN 

{Obtain only after all elements of your plan have been evaluated) 

I judge the plan as i whole to be of 

(circid one) 



School Supervisor 
College Supervise]^ 



Content Specialist 



ACCEPTABLE - OUTSTANDING 
QUALITY ^ <2UALITY 



Field Test Format 2 
Experimental Elementary Teacher Education Program 
Oregon College of Education 
December 1972 
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LESSON PLAN EVALUATION FORM 



Student's Same * ^^son Number 

Have the evaluators that check your plan initia! each of the items listed that meets with their approvaK If the treat- 
ment of an item is thought to be outstanding, have the cvaluator draw a circle around his or her initials. Be sure to attach 
this sheet to your lesson plan. 

EVALUATORS OF THE PLAN 



ELEMENTS OF THE PLAN 



OBJECTIVES 

Are the learning outcomes expected from the lesson clearly 
stated? 

Are they appropriate and worthwhile outcomes; givcA the 
characteristics of the pupils to be taught? 

Are the indicators that are to be used as evidence pf suc- 
cessful outcome achievement identified? 

Are the procedures to be used in obtaining evidence of out- 
come achievement identified? 

ADAPTING OBJECTIVES TO LEARNER 
CHARACTERISTICS 

Ar«^ there provisions for modifying the objectives of the 
lesson to meet individual pupil characteristics? 

SELECTING INSTRUCTIONAL MATERIALS ANL 
PROCEDURES 

Are the instructional materials to be used in the lesson 

clearly identified?. 
Are they appropriate to the learners to be taught and the 

learning outcomes to be achieved? 
Are the organizational and instructional procedures to be 

used in the lesson clearly identified? 
Are they appropriate to the learners to be taught and the 

learning outcomes to be achieved? 

EVALUATION 

Are there provisions for determining where pupils stand 
with respect to the desired lekrning outcomes of the 
lesson before it is presented? 

Are there provisions for eedback to pufJtts about their per- 
formance during the ti. le the lesst^n is being presented? 
s \re there provisions for determining where pupils stand 
with respect to the desired learning outcomes of the lesson 
after it has been presented? 

PLANNING NEXT STEPS 

Is there some indication in the plan of what would be done 

next with the pupils if the learning outcomes expected 

from the lesson materialize? 
Is there some indication in the plan of what would be done 

next with the pupils if the learning outcomes expected 

from the lesson did not niaterialiic? 

MATCHING INSTRUCTOR. LtSS(>N AND CONTEXT 

Does the lesson as planned appear to b * feasible and ap- 
propriate to the school setting in which it is to be pre- 
sented? 

Does the lesson as planned appear to* be feasible and ap- 
propriate to the student who is to present it? \ 



School 
Supervisor 



College 
Supervisee 



Content 
Specialist 
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Field Tcsx Format 2 
Experimental Elementary Teacher Education Program 
Oregon College of Education 
December 1972 v 
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DEMONSTRATION CONTEXT: LESSON PREPARATION AND PRESENTATION 

SUMMARY EVALUATION OF LESSON PRESENTATION 



Student's Name 

We judge each of the lesson 
elements that folloxy to be of - — ^ 
(circle one) 

OBJEGTIVES 

ADAPTING THE LESSON TO 
LEARNER CHARACTERISTICS 

QUALITY OF PRESENTATION 

Classroom Management 
(Transition, Termination, and 
Student Attention) 

Instruction 

(Materials* Procedures* and 
Organization) 

EVALUATION 

Preassessment 

Feedback during the lesson 

Feedback after the lesson 

Achievement of desired learning 
outcomes 

PLANNING NEXT STEPS 

THE FIT BETWEEN INSTRUCTOR, 
LESSON. A-ND CONTEXT 



Lesson Number 



UNACCEPTABLE 
QUALITY 



ACCEPTABLE 
QUALfTY 



Dale 



OUTSTANDING 
QUALITY 



J 



-I 



School Supervisor 



College Supervisor 



Student Presenter 



Content Specialist 



Student Recorder 



Field Test Format 2 
Experimental Elementary Teacher Education Program 
Oregon College of Education 
December 1972 
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Eslimaling Costs of a Competency-Orientation 

By i 

V BRlJCi!: IK. JOYCK 



How do \vc buiUl a Ci>i\ipci Miey-orioiUod cortitication 
and training system? Personally, I could never gjj. 
through all ihc necessary complicated steps, althougli 
there are people here who could. I'm much more inter- 
ested in trying to find out what leaching skills will pay 
off; whether they have to be idiosyncratic; whether any- 
body can even train us all in the same on«*s. Its con- 
ceivable that we ail may not be* capable the same 
ones; ihaybir in the long riiri we will have to do our 
own thing* as teacher competence, in the end, may turn 
out to be personal — i>r it may be external and train- 
able. Also, teaching »v elusive. You can measure 2,000 
reasonable-soufiding competencies and not have caught 
the essence of teaching. So far the research hasn't vali- 
dated many competencies, but Pm not prepared to give 
it up. 

To determine competence, we ]|Ui> have to face 
some rough questions. For example, we'ri not even 
sure that as teachers we should be playing the roles we 
do. I keep critizing th : role of the self-contained class- 
room teacher mostly because it nearly killed, me. I had 
to go into college teaching to keep alive, but I .simply 
couldn't teach six things to 36 kids who were that dif- 
ferent from each other. 1 didn't have the capacity for 
the job, rfnd I had to become a profes.sor in oruc; to 
survive. I'm not sure that I want to .spend any time 
finding competencies for the present roles teachers 
play. We might do much better to design better schools 
and find out what competence we need to make them 
work. 

I know nothing about costs. OK? 1 couldn't cost-ae- 
count a major program, but last summer, some of the 
New York State Education Department people asked 
me to help them estimate how much a competency 
program would cost to develop. 1 agreed to do it, and 
I wrote tnem the following f^aper. 

Largely because of the massive research and devv^- 
opment effort entailed if competency-based training 
and certification are to be achieved, it has been sug- 
gested that development and implcnjcntation be cen- 



tralized. An alterntitive to aceommovi;»te k>eal needs 
and bring about a broad base of pariicipation in devel- 
opment is that the competency-orientation should be 
required uind relatively small local consortia (probably 
made up in u<!Gh case of a few school districts and col- 
lege s) jihi yfld determine competency criteria, develop 
programs, and' implement the programs and the certifi- 
cation procedures. The latter option is attractive be- 
cause it promises to involve so many people and to in- 
sure relevance to local school 4ieeds. 

The centrally mandated system is inacceptable on 
both political and substantive grounds: politically be- 
cause it would give":considerable power to a fc^v per- 
sons (competency-oriented^ cer^fication and training. 
much mote likely to affect whQ will actually be perhiit- 
.ted^tA.lteaeh than course-based certification and train- 
ing, however centralized), and substantively ffccause 
not enough is yet known about the identification of 
teaching competencies to permit anyone to' develop a 
widelv mandated set of compet<:ncies for any particular 
eates? l y of teachers with any great confidence* that 
those competencies will standi up. 

Centering development a.ound many local centers is 
acceptable as a process, but is not without serious 
problems. The more twal centers there are, the more 
difficult it will be to organize teani,s of sufficient exper- 
tise to develop really strong training and certification 
systems. Major universities,*'working with strong school 
districts, have had trouble doing this. Yet, strong de- 
velopment teams arc ncvessary. High. quality in a com- 
petency-orientation is essentia' because both certifica- 
and training procedures will be more powerful and 
mistakes (such as emphasizing trivial ebmpetencies) 
will be magnified. 

Many districts and smaller colleges are unaccus- 
tomed to freeing personnel for dcvelir)pment, but a sus- 
tained effort by a large team is essential if the task of 
identifying competencies, organizing training ancf 
ses.smcnt, and implementing i ertifuj.tion proceciu»cs js 
to be done effectively. 
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Without coordination and quail, v control, broad par- 
ticipation in the program dcvclOi»nicnt could be an 
endless and ineffective process. 

Purpose • ^ 

In this paper an attempt is made to identify the 
tasks of creating a competency-based v*acher training 
and certitication system and to ostimai : the costs of 
completing the tasks adequately. In addi ion, a base is 
provided for estimating the effect on co ts of several 
possible options for organi/.itig the process of develop- 
ment and implementa>imj. This can be used as a base 
for estimating cost when several possible options for 
organizing the process include broad local input into 
the system, lo estimate basic technical costs an esti- 
mate is nuule of the cost of d.*^ ♦^lopment by a Lompact,"^ 
independent team of experts who would sut^A, ^accom- 
plish the technical tasks of producing a prototype • 
tftiining-ccrtification system. Ihis option (referred to 
as process option A) is not a realistic ehoiie, but 
yields a fairly reliable cost estimate from the technical 
point of view/ which permits an estimate of the cost if 
several possible organizational processes were em- 
ployed; e.g., Htrictiy local development (process B)» 
kKa1 development with^ central technical assistance and 
development services (process C), and a general >tate 
system with provision for local options (process D). 

4% 

Source of Cost Estimates 

How do we arrive at the eosis of the tasks and sub- 
sequently, the eosts of the proeess options? There are 
three sourecs'of our estimates. One is the experienee of 
the Bureau of Research Tcaeher Education Projeyn, es- 
pecially the costing priKcdures included in their feasi- 
bility studies. Tlje second is the cost of developed per- 
formance-based materials such as tho.se in "Materials 
for Modules" (appendi.\ B). These two sources can 
help us determine the base costs represented in process 
option A (the least desirable priK-css option in terms 
of creating a statewide training-certification system). 
I he third source is the least reliable and represents es- 
timates of the probable increase in the base cost due lo 
the greater complexity and duplication of effort of the 
other process options. 

Technical Tasks and Process Options ^ 

Certification and training are interrelated parts of 
the same system which serves as the basis for piescr- 
vico teacher education, inscrvicc teacher ciiucalion. and 



the certifying and diagnostic procedures related to both 
of them. 

The purpose of the development^of competency-ori- 
ented systems is to unite preservice education, initial 
certification, inscrvicc education, and continuing or ex« 
tended certitication around a system of teaching com- 
petencies which form the goals of preservice education, 
the standards for initial and extended licenses, and the 
basis for diagnosis of performance of inscrvicc teachers 
and prescription of inscrvicc training. It is probably 
not jHwsible or desirable to separate licensing or< train- 
ing at the preservice or iaservice levels under the coin- 
petency orientation. 

The ejitire system for competency-oriented certifica- 
tion and training depends on the creation of four 
interrelated storage and retrieval systems plus the orga- 
nizational and communications networks necessary to 
create and implement them. The creation of these sys- 
tems and networks constitutes the technical side of the 
development process. The creation and management of 
the organization necessary to create the systems consti- 
tute the process side. Both technical and process sides 
present options which affect costs greatly. 

The Technical Side 

The technical side of competency-based certification 
and training involves the creation and validation of 
four systems. These arc as follows: 

A storage and retrieval system of the specifications of 
teacher competency. 

Since teaching is complex the numbef of competen- 
cies which are likely to be specified is large indeed. 
The Bureau of Research Models averaged between 
2,000 and 3,000 competency specifications and these 
represented efforts by single fified institutions or 
small con.sortia rather than statewide consortia of di- 
verse institutions and representation. It seems reason- 
able to suppose t^t the number of competencies will 
increase as the political base for establishing them is 
broadened. Hence, when teacher asisociations, repre- 
sentatives of school administrations, the public and stu- 
dents all contribute, as well as expert teams from uni- 
versities and state departments of education, the 
number of potential competencies of a teacher will be 
large and the process of identifying the most important 
competencies may be complex. 

.i stimif^e and retrieval system of mediated instruc 
tional systems and afient-mediated components de- 
signed to produce tiie competencies. 



This ' storage and retrieval system represents the 
means of teaeher edueation. The number of items 
stored as instruetional systems will be equal to or 
somewhat greater than the number of eompeteiieies 
whieh are spceifted* Also, the extensiveness of eaeh in- 
struetional system is mueh greater than the speeitiea- 
tion of any eompeteney. Thus, the produetion of soft- 
ware to fiinhis system Vill be an extcftsivc task. Put 
more simply, it will be diftieult to speeify eritieal eom- 
peteneies, but developing, the program a*lements to 
achieve them will btf"cven more eomplcx. 

A sysiem of axsessm^^m devices designed to determine 
the effects of the .nAtructional systems, and agemy- 
mediated components and to measure th^ competencies 
specified in system A . 

<^ 

Without an assessment system eompeteney-based 
eertitieation would be impossible as would be the as- 
sessment of the effectiveness of program elements. 
Properly organized, , the assessment system provides a 
diagnostic profile of the teaeher candidate and provides 
also the means for tracking his progress to determine 
when certification should be granted. In the c; • of in- 
service teachers^ it provides the means for diagnosing 
the state of their competency and relating them to the 
instructional systems to be used. in inscrvice education. 

A management system for interrelating subsystems /. 
2, and 3 above. 

The magnitude of the three other systems requires 
the use of a contemporary management system for di- 
agnosis, prescription, trackit^ and progress, and pro- 
viding feedback to teacher candidates, teachers; and 
program and licensing managers. Without a complex 
automated management system individualization or 
personalization would be impossible in a program and 
so would be implementation, for the complexity of 
specifications and training devices is so great as to de- 
feat any presently available option for program control. 
Without a modern information system chaos would 
^surely ensue, at the point of implementation. It is po.ssi- 
ble to imagine a statewide managCmeQj'5);.stcm which 
individual teacher training institutes would iine to guide 
them in the identification of competencies, and they 
might use a central system to withdraw instructional 
sy.stems and a.s.se.ssment devices for their particular 
training program. 

.rhe research and development effort needed to pro* 
duce the four interrelated systems is enormous. I hcy 
represent the lost of the substance of competency- 
based teacher education and licensing. 



93 

ft 

The Process Side 

If it is desired to have participation by all relevant 
groups in the creation of the competency-based teacher 
edueation and certification system then suitable com- 
niunications networks and organizations have to be set 
up to permit teachers, .students, teacher organizations, 
representatives of the public, subject matter experts, 
experts on teaching, and representatives of .state de- 
partments and colleges to participate in competency 
identification, the selection of instructional systems, 
and the creation of the as.ses.sment sy.stems. The proc- 
ess of creation can reasonably be divided into three 
levels; specification of the systems, development, and 
implementation. It is possible to imagine a state or na- 
tionwide network providing participation by all rele- 
vant groups in the specification level of the creation of 
the basic .system. Development will proba]^ have to 
be organized at a few major centers, funded to bring 
together talent to create the operating syt^tems. It is 
possible to imagine a network stretching across the state 
so that many Jocal units might contribute something 
to the development. However, development is an ex- 
ceedingly complex task^ especially if the products are 
to be tested as they are developed. Thus, a few major 
centers probably will do most of the actual production 
of the instructional systems and testing device^. How- 
ever, broad-ba.sed groups can evaluate the products, 
and a representative group can monitor the' entire 
operation. 

Minimum, General, Statewide Competencies 
and Extended Local Competencies 

It is possible to use some of the critical competency 
specifications for minimum statewide certification — 
these could represent minimum standards for preser- 
viee and in.service training. A much larger number of 
specifications might be prepared representing local 
needs. These would be used by Individual teacher 
training centers for iraining purpo.ses and for determin- 
ing local certification in addition to state certification. 
Imagine, for example, a network of field centers repre- 
senting the local school districts, higher education insti- 
tutions, community representatives, teacher a.ssocia- 
t^ons. etc. Suppo.se that there are about 20 of th^ in 
various parts of the state. They produce competency 
specifications and agree on tho.se which will be used 
for minimum specifications for any given type of 
teacher ( elementary, secondarj', special education, 
etc.). Rach of (he local centers, however; would pro- 
duce specifications which were deemed imp^)rtant by 
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the people in that ecnler but whieh were not seleeleil 
by statewide representatives as essential for statewide 
eertitieation. I hese additional specifteations ni>netheless 
could be stored in the sivcitiealions storage system and 
instructional systems could be designed to achieve 
them and assessment systems to assess them. Ihus a 
complete statewide system, if it is to take care of local 
needs, would have to be much larger than the system 
which would .produce only slate requirements. Clearly 
the greater the participation of inlcresteil parlies and 
the more numerous the local centers, the more costly 
the creation of the system. !-'rom the point of view of 
organization and communication and also from the 
point of view of the size of resulting system participa- 
tion and with provision for local options costs will in- 
crease. 

This cost would-be nothing like the cost of asking 
each local unit or center including several local units to 
create their own competency-based teacher educatioii 
and certification system. The cost of building any 
stronu. competency-based program with local options 
will be $10 to $12 million (and that is probably the 
minimum which should be anticipated). It would cost 
^ $5 to*$7 million for each local unit to create its own. 
systenft Thus if there were 10 systems in the state each 
developed by local units, the cost ..would be in the 
neighborhoiKl of $60 to $70 million. It w^juld seem far 
more reasonable- in cost to create a statewide system 
that had provisions for special lothl needs and inter- 
ests. 

Technical Tasks and Process Options 

The four tasks involved in the creation of a compe- 
tency-based program, plus the development of a certifi- 



cation system, represent the five tasks required to 
activate a statewide cpmpclcncy-orienicd training and 
certification system. I he cost of each of these will vary 
cpj^iderably depending on the process options which 
are selected. In table t, four process options are de- 
picted over the five tasks. . ' .. 

PriKcss A (deve1<iping one complete prototype sys- 
tem ) is the lei»=t costly option in terms of dollars, but 
has the disadvantage that its statewide acceptance 
would not be high, in all likelihood. 

Process B (development of complete system by sev- 
eral local centers) would be very costly. Ihe cost of 
each complete system would be higher than that of a 
system developeil by an "expert" team (! estimate that 
the cost would be 50 percent higher) and that would 
be multiplied by the-number of local centers. 

Process C (local development supported by a few 
development centers) would be somewhat less, costly 
than B because the development centers could redtfcc. 
duplication of effort. 
^ Process © (development of a statewide systein with 
local options) would be much higher than A but much 
lower than B or C and would result in an acceptable 
statewide prix:ess and plan. It appears to be the best 
cost option. 

"Process A is least costly because most compact. It is 
useful, however, for cost estimates because it can yield 
i; base cost which can be multiplied by factors repre- 
senting the increased complexity and duplication of ef- 
. fort required in the other options. 

Process B results in several comparable systems. It 
gives local needs the fullest play\ but the use of nonex- 
pert teams escalates the cost of each system so that 
each local sy.stcm would be very expensive and the cost 



Table 1. Cost Factors of Technlijgl l asks by Process Options 





A. Prototype 
hy ExiK'rt Team 


B. Local Consortia 
as Focus 


C. Local Consortia 
Plus Development 
Assistance 


^ D. Statewide 
Syste^i With 
Local Options 


I . Competency System 


I 


1.5 

(no. of consortia) 


1.25 

(no. of consortia) 


2 


V 2. Instructional System 


1 


1.5 


1.25 


. -> 


3. Assessment System 


1 


1.5 


1.25 


2 


4. Management System 


I 


1.5 


1.25 


M 


5. Certification System 


I 


1.5 • 


1.25 


2 



BEST COPY aVmUBU V 



within the state would be the cost of each system times 
the number of local systems. 

Prwess would be a bit less than B due to some 
cenirali/ation of effort 

PriKcss D, providing for wide local participation but 
central coordination to reduce duplication ot effort, 
might only require about twice the investment of proc- 
ess A- 

The Base Cost of the Technical Tasks 

The Uuicau of Research Mixlels, especially the^M- 
sibility studies, provided cost estimates of many of the 
basic technical tasks. These are supplemented by the 
actual Vosts of develo;^ing performance-based training 
materials. I he Florida State University and University 
of Wisconsin estimates provide data especially relevant ^ 
to the types of activities which New York appears to 
require. ^ 

General Costs 

The Florida State team attempted to identify the 
various costs of startji^ and implementing a competen- 
cy-based teacher education program that would pro- 
vide conrtprehcn^^ive clinical training. That is» they were 
dealing with the professional components rather than 
the liberal urts components and the others that might 
contribute to the education of a teacher. They planned 
abi>ut 300 program units. They estimated about $2U 
million to develop the units, about $75»000 to develop 
and test the entry diagnostic system, about $400,000 
to develop and equip the computer management sys- 
tem, and about $200,000 to carry on a faculty training 
program, making a grand total of a little less than $3 
million. 

My personal view is that this estimate is conserva- 
tive. For example, the faculty training program is not 
costed very effectively by years. It is worthwhile not- 
ing, however, that they probably assume that many of 
thfe faculty would be persons who are Involved in the 
devcljjpment and who would, in the course of develop- 
ing the materials/ train themselves to carry on the 
kinds ot activities they would need to engage in as fac- 
ulty members. It is worth noting, that they did not ex- 
pect to implement a program in I year or 2 but rather 
expected to take 5 years fpr thp development and im- 
plementation prtKess. 

I think this is a very fair estimate of time. Develop- 
ing, testing, and integrating really significant, compe- 
tency-bAsed instructional systems will be a 
time-consuming and very expensive task. Furthermore, 
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if one were to go to a competency-based licensing, one 
would not want to do so until he had determined the 
general effects of a competency-based teacher educa- 
tion program. In addition to the time for **start up*' 
that was estimated by .Florida, it might take 4 or ^ 
years before enough people had been through the pro- 
gram observed as teachers to lay a data base for revis- 
ing the certification procedure. Thus, to move from the 
present teacher education program structure to a com- 
petency-base could be somewhere between 5 and 10 
years, probably in the upper end of that range. Process 
Options would not affect time much, because develop- 
ment of materials is the chief consumer of time and 
the process options would affect only the other tasks. 

The above costs did not include competency identifi- 
cation, which averaged a bit over $100,000 for each of ' 
the Bureau of Research studies. Since the Bureau of 
Research teams generally feel that the specification 
would have to be reworked before development could 
begin, probably another $100,000 would have to be in- 
vested in this task. 

The Cost of Data Processing and 
Management System 

It is not possible to imagine competency-based edu- 
cation on a large scale unless it is supported by a com- 
puterized inanagement system. The eventual programs 
may be somewhat less complicated than those envi- 
sioned in the Bureau of Research models, but units in 
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a compctcncy-bascd program will amount to a very 
large number. (Probably move than 2,000, still under 
the average number in the Bureau of Researeh pro- 
grams.) In relating 500 to 1,000 teacher candidates to 
a program containing that many elements requires in- 
formation access, the coordination of support mate- 
rials, and the coordination of faculty of very great 
complexity. 

The cost of creating such a management system will 
be considerable. The University of .Wisconsin did a 
very thorough job of . idejUifying the elements of a 
management system and the costs*^-t4, maintaining a 
staff sulficient to operate the system. They thought out 
a system which would have the capacity of handling 
over 700 students in an on-campus setting. They esti- 
mated an annual cost of amund $712,000 to maintain 
the system (about $1,000 per student.) The rental of 
hardware for the system would be in the neighborhood 
of $350,000, and the cost of the data processing staff 
about the same. However, such a management support 
system would have the capability of serving *i greai 
many more students at relatively low additional cost. If 
a state the size of New York decides to move into 
competency-based education, it would seem wise to de- 
velop one or two prototype management systems. If 
carefully djjsigned they could 'accommodate the pro- 
gram units for a number of programs of different 
types. This effort would reduce substantially the cost of 
developing and maintaining management systems and 
institutions throughout the state. However, it is very 
h^fd to imagine that even the greatest amount of shar- 
ing among the centers for teacher education would re- 
duce the cost of maintaining a management system to 
much under $500 per student. The system, however, 
would permit an individualization of instruction, pro- 
gram planning, and assessment far beyond the capacity 
of any present teacher edueatioi^ program.. Also the 
system envisioned by Wisconsin is a multimedia system ^ 
using computer system instruction, instruction through 
motion picture, television tapes, audio tapes, and pro- 
gramed units providing a variety of instructional modes 
which very few instructional settings presently offer. 
Thus the utility of the management system is general 

it handles a large number of program elements for a 

large number of students and increases the type as well 
as number of instructional options. 

On the Requirement of an Automated System 

It is worth noting that none of the Bureau of Re- 
search models was conceived without the assumption 



that it would be possible to operate a competency- 
based program without an automated management sys- 
tem. To relate 500 students to 2,000 instructional units 
in sucn a way that students have instructional options, 
are assessed and. made aware of results, and relate to 
program options in terms of developed competence, a 
management system is simply ftccessary. 

The expectation should be that part of the develop- 
ment t»f the competency-based system involves the cre- 
ation of management systems. No program plan should 
be accepted that does not include the provision for the 
devek»pnjent as such an automated system. 

Base Cost Estimates 

Under process option A, the costs of the tasks 
would seem to break down as follows; 

I^veloping the Competency System $200,000 
Developing Instructional Materials mtnimuftj $2,500,000 

(including Assessment Devices) 

Developing Diagnostic System $75,000 

Developing Management System $400,000 

Annual Cost of Maintaining Management $750,000 

System (to serve one program) 

These can be multiplied by the factors in table. 1 to 
develop very rough estimates of costs for the other 
process options. 

The experience of developers in recent years adds 
some specificity to the cost estimates and suggests cau- 
tions. * ■» 

Development ot Specifications 

The post to the United States Office of Education of 
having nine models developed by relatively expert 
teams in a very short period of time (about 8 months 
— probably not a suflRcient period of time to do the 
job right) was over $3V6 million. Relatively inexpert 
teams working over a longer period of time will i^roba- 
bly be much more costly. The cost could be reduced 
by havif!f*?l statewide organization in which state and 
liKal teams take responsibility for certain areas and the 
whole effort is coordinated. Local participation, espe- 
cially to lay down initial preferences and to monitor 
the results, could be encouraged. 

Tlie Development of the Training System 

Thus, the training system area is where the real cost 
actually begins. The Bureau of Research project esti- 
mated between $5 and $7 million to develop their sys- 
tem, using a single team within a consortium or withi* 



a single Institution of higher edueation linked to loeal 
seluH>! distriets. The broader the base of partieipation 
and the more eompLw the model of teaeher edueation, 
the higher will be the eost. 

The eost of the mini-eourses developed at the I-^ar 
West Laboratory is illustrative. *I*hc Far West Labora- 
tory, uses an extremely ettieient development team 
whieh ereates and tests its produet using standard re- 
seareh and development proeedures. Ihe teams arc 
well trained and stay together for long periods of time. 
They do not have responsibility for a wide diversity of 
produeis but poneentrate on the partieular type of in- 
struetional system known as the mini-eourse./rhe eost 
of development of eaeh single mini-eourso providing 
about 30, hours of instruetion to a teaeher on one 
teaehing skill is over $I()(M)()(). At leaehers College, 
Columbia University, a series of 4iix instruetional sys- 
tems designed to teaeh three teaehing skills and three 
strategies was developed at a eost of a little over 
$1()0,0()(), or about $T6,()()() per instruetional system. 

It is probable that a eomprchetniivL teaeher training 
program at the preserviee level will eontain at least 
200 instruetional systems (as in Florida's estimates). If 
that is the ease and the development cost of $15 or 
$16 thousand found by the Columbia team turns out 
to be an aeeurate one, the estimates from the Bureau 
of Research teasibility s»»»*<»«'*i *\rt: aOOUi on lai^wi. 

rhis will be the eost of developing one model of 
teaeher edueation by one centrally organized team. As 
indicated earlier, the greater that development is de- 
centralized ( process options B, C, D) the greater will 
be the cost unless the teams specialize and do not at- 
tempt to build the entire range of pcissiblc instructional 
systems. 

What might be done would be to develop a master 
.plan of development and subcontra?n pieces of the de- 
velopment (process I>^. However, it has to be remem- 
bered that there are very few places presently in the 
state where much developmental productivity could be 
expected for some time to come and it would take cen- 
ters a while to tool .up. Therefore, it seems wise to 
make the assumption that a few places in the state 
would have to be funded for major development ef- 
forts. 

Further Notes on the Assessment System 

The picture is somewhat different with respect to the 
assessment subsystem, rhe developers of the instrue- 
tional systems could create many of the assessment 
measures for specitic eompeicneies as a part of their 
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developmental effort. This would probably be the most 
etlicicnt way of developing the specific measures and 
w;k the procedure pri>posed by the Bureau of Re- 
search team. However, in addition to the measures of 
specific competency, as they would relate to the in- 
struetional systems, one needs measures of the general 
competency of the teacher or situations in which he 
can bring together a group of competencies and perform 
in a more cfTcctive way. It was recommended in sev- 
eral of the Bureau of Research proposals that teaching 
laboratories be used for this; that situations be set up 
in which a tejicher could be brought together with a 
small group (jif students. He would teach for an hour 
or two» while his ability to set objectives and teach, 
iv-^'^ the gains in pupil knowledge were assessed at the 
same time. Some kind of assessment system like this 
would probably be desirable. It is unlikely that rating 
blanks or rating systems of observing and rating 
toacher class behavior in the teacher's own classroom 
will prove to be feasible. For more than 50 years per- 
sons have been trying to develop such measures, with 
notable lack of success. 

Saving Money by Sharing Materials 

The technica? nature of the competency-oriented sys- 
tem niaki*s it nnnsmUv H<*npnH«nt on tVi.* 
o^mpetency-based software. Specifying the competency 
necessary for any particular teacher role, building ap- 
propriate instructional systems, creating reliable assess- 
ment systems, and implementing a program all require 
extensive research and development efforts and .major 
changes in procedures. Statements of competency, in- 
structional, systems, and assessment devices are all ei- 
ther wholly (as in competency specifications) or partly 
expressed on paper, film, television tape, etc. Without 
a very large development effort, there will be no imple- 
mentation of CBTE. 

Under some circumstances, a solution to the need 
for development might be to fund one or two agencies 
to create complete systems and disseminate them. 

In this case, however, the actual implementation 
must be closely allied to UkuI needs and local person- 
nel must participate substantially in the whole process 
of determining directiol^ for change, defining teacher 
roles, and selecting and implementing training systems, 

I'o support liKal efforts without cither controlling 
them or engaging in a ruinously costly duplication of 
development costs in every local agency, a large na- 
tional storehouse of competency-based prinlucts should 
be developed. From this storehouse, local agencies can 
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draw much of what tht7 will need to build their 
compctencyTbuHcd trailing system, while developing 
the remainder locally. Fortunately, agencies across the 
Nation have ^een producing materials which can get 
the storehouse off to a good start. 

The area of ledding instruction provides a good 
example. There presently exist a number of approaches 
to reading 'and a number of systems for acquainting 
teachers wHh them and training competL%ties relevant 
to tbost approaches. A local agency can draw from 
these in order to build the reading component of a 
program to train preservice or in.servicc teachers. 
This situation is duplicated in several other areas. 
Qje developers represent a range of agencies, mony 
of<^wiiich are wholly or partially supported by Federal 
funds: 

1. Research and development centers and re- 
gional laboratories arc available. 

2. Curriculum projects in niany areas, including 

ubject areas, approaches to the education of 
young children, special education, correction, ca- 
reer opportunities, etc., are in c)tistence. 
•3. N.C.l.E.S. projects, mcluding the Teacher Cen- 
ters, Texas Education Agency, Protocols proj- 
ects, Florida State, are funded. 

4. Teacher Corps programs have included 
development officers for the last 2 years and 

projects huvL loucIw 
uted products and processes. 

5. Colleges and ^ronsortia which are in the process 
of reaching for the competency orientation have 
pi-oduced many materials. 

f he result i,s a beginning to a national storehouse — 
provided that the products are brought together, made 
visible, and demonstrated in preservice programs and 
inservice teacher centers. More products are needed, to 
be sure, and many need transformation and testing, but 
a considerable start has been made and should not be 
lost. Many agencies are presently operating from 
scratch unaware of the considerable array from which 
they could draw. 

What is necessary to make the storehou.se useful? 
First, potential users need to know what exists in the 
storehouse and the potential use of each product. 
(Presently the Florida Catalog of Competencies, the 
Florida Catalog of Competency-Based Materials, and 
**Materials for Modules,** prepared at Teachers College 
for the 1 eacher Corps, list many materials.) 

Second, users need to know how materials actually 
work— what an instructional system or assessment de- 
vice can be expected to achieve and what it takes to 
U.SC it. 



I bird, some materials need further development to 
make them adaptable to many local situations— their 
transportability needs to be improved. 

Fourth, users need to experience competency-based 
pri>grams, and IVacher Centers can operate with prod- 
ucts largely drawn from the storehouse. 'Ibis is ex- 
tremely iniportaiu. The storehouse has now reached 
the point where energetic designers can, by using the 
products o*' others and developing some products 
thcnVselves, actually operate programs and Teacher 
Centers which would l)e.at least three-fourths compe- 
tency-based. 

The surprising fact is that there is presently avail- 
able much more software, especially^in terms of in- 
structional systems, than could possibly be used in a 
2-year, full-time, teavjber preparation program. Not all 
of the material is of high quality—and developm**nt 
has been n^issing in some important areas — but careful 
selection from the storehouse, combined with local de- 
velopment, will enable program implementation fairly' 
soon, provided that the available materials are dis.seml- 
nated and demonstrated and if program ^'models and 
Teacher Centers using the storehouse are also estab- 
lished as demonstration centers. 

The Nature of the Storehouse 

Most of the ^products arc in the form either ot m- 
structional systems or smaller units, generally called 
modules. Generally they consist of a competency speci- 
fication, a set of activities and supporting mediated" ma- 
terial, and an assessment device. Materials are avail- 
able in at least the following areas. (See following page.) 

Imagine, for example, a p'ogram to prepare teachers 
of the'social studies. 

It might include competency And adapt the foUowing mate- 
in: rials: 

Parsons* Ckiidcd Self-Analy- 
sis or Mini-course on 
^ Flanders Sysiepi 



( ! ) ihe study of teaching 

(2) basic educational 
psychology 

(3) basic instructional 
design 

<4) curriculum alternatives 
in siKial studies 

<5> basic teaching skills 



(Cirinder pi'ogramT 

Vincel cPopham) 

l.l.U. (har West Labora- 
tory) Teachers College Units 



Mini-courses 
l eachers C ollcge limits 

(f>) basic teaching strategies l^rcsentalional skills (Gen- 
eral l earning Corporation) 
Teachers ( oUege Units 
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Using ♦ the alrcad) -developed materials (ineluding. 
eumpetency specifieations. instructional materials, and 
assessment dcviecs), one could easily organize a fac- 
ulty to Implement a competency-based program to pre- 
pare teachers of the swial studies. Most teacher educa- . 
tion agencies would probably prefer to create some of 
their own materials, but to develop all they would need 
would probably require an investment of $400 or $500 
thousand, whereas if they draw from the national 
storeluiuse the cost becomes feasible* 

A Teacher Center 

To build li I eacher CenteY would involve much the 
same process. A 'i Lacher Center should offer teachers 
three types of service: ♦ 



^ « « 
K Teachers Opiion. When the teacher selects com- 
petencies according to his interest ot self-diagno- 
sis. ' . * • 

2. Exterml Diagnosis ^Option. Whei> the tcachpr'^ 
• peers or supervisors determiiis ^*^t he has a 

need by analyzing his teaching. 

3. CuriticHhtni "Thrust Option. When a team, 
. school, or agency inaugurates a new program re- 
quiring new competencies. ^ A-- 

l o create such a center one needs to offer teachers a 
great many options, which can l:>e ^electiorf on any of 
the three service bases described above. For example, 
instructional systems could be * clas^fied on a basis 
somewhat like the Array of Products and made .avail- 
able with a microtcaching laboratory. Thtfs: 




□ 




□ 


Skills 


□ 


□ 


□ 


* > 

• Strategies 


□ 


□ 




Curriculum 


□ 


□ 


□ 


Psychology 






□ 


Btc. 

4 



Teacher Centers could be designed around the roles 
of the tejicher. A good many of the existing products are 



Ssuited 0"the various roles. For example: 




General Competencies 
"Study of Teaching 

instructional Design 
Basic Teaching Skills 



Role Related Instruction 

Skill Builder 

(I PI Training System) 

(SRA System in Reading) . 

Productive Thinker 
(Synectics Material) 

Community Leader 
(Teachers College Units) 

Counselor 

(Micro-counseling Skills) 
Academician 

(McCrel Units, University of 
Texas Units, Teachers College Units) 



More .than enough matena^ presently exists in the 
storehouse to permit the creation of teacher Centers 
such as this, t A)upled witfr the types o£ workshop een- 
ters.used in the English Teacher tenter}>, they provide 
a gxmi many possibilities for teacher self-training. 

Cost and Quality 

Even if a local program built all its own staff, 
I think CB TE is dirt cheap even at the level of cost I 
havi^ estimated. I thmk it is extremely inexpensive. 
What is really expensive is what we are doing now.' \)'e 
pour hundreds of millions of dollars into ineffective 
teacher; education programs every year. That's ^almost 
pure waste ' The inefficiency and waste of ihe present 
system'is just terrible. ' ^ ^ 

A great deal of the cost of development will be in- 
.Icind costs. View a teacher education program as a de- 
veloping nation. You have to develop capital somehow 
and there are a couple fdrms of capital: one is ideas; 
the other, software. You Invest faculty time, in order to 



get these. We' simply heed to put a greater proportion 
of staff costs into development than we have. 

What bothers , me about the stangc thJ|t it will be 
easy and cheap is when development simply becomes 
modulari/ing the . present stuff of. teacher educatiort. 
I hat /.V easy and cheap byt it jlocs.n^t result in much 
improvements' 

i/have never personally experienced a major univer- 
sity (jnaybe Houston, mfnybe Toledo) that was capable 
of doing the whole development job by itself. The Na- 
tional C\)nsortium is essential for just pooling the tal- 
ent to do the job. For example, without the 
Mini-courses produced at tjie Far )\'est Lab and gther 
materials produced by dozens of others, i wowldn^t 
dream of rhc type of program we run 'at Columbia al- 
though W^* build a lot of instructional systems our^ 
selvei. We must share materials if we -are to keep costs 
m bounds and draw on each other for program id^as. ^ 
CB TE should be a national development effort with a 
myriad of local* variations. 
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Funding Competency-Based Teacher Education 

By 

IIKRBKRTHITE 



Compcicncy-bascd teacher edueation costs diflFer 
from the costs of standaid teacher education because 
tfie two programs differ. I'here are differences in the 
required performances of students and also in the roles 
of faculty. Program materials are different and so are 
♦Assessment procedures. The sources of funds for 
CBTE, however, are not different from the sources for 
traditional, oi standard, programs. The same sources of 
revenue for higher education which ace always in short 
supply must implement the CBTE programs. There- 
fore, the analysis of costs for GBTE is most meaning- 
ful when these costs are expressed in terms of funding 
formulas for other higher edueation programs. 

Among the program variables of CBTE which are 
reflected in differi^ costs from standard programs are 
the following: * 

1. Instructional materials, units, and techniques 
are individualized. If the criterion of CBTE is 
the success of the individual student of teach- 
ing, then CBTE materials must allow for differ- 
ent rates and different options Dy these individ- 
ual learners. These materials constitute a higher 
cost factor than standard program materials. 

2. For the most pail, the CBTE faculty wo. in a 
one-to-one relationship with students— as 
counselors, evaluators, explainers, analysts. The 
changed role of faculty has a higher cost than 
faculty in the traditional lecturer role. 

3. The process of admission to CBTE appears to 
. be a more complicated and hence a more costly 

process than admission to standard programs. 
Prerequisites for success, which should consti- 
tute admission criteria, seem to be more diffi- 
cult to assess than the usual examinr^tion of 
students* grade-point averages. 

4. Complex management systems may be a signifi- 
cant cost item. 

5. Students in the CBTE programs are usually 
full-time interns or observers or work over 
varying periods of time with individualized 

^ These 10 factors are described in a paper given for the 
f eacher Corps in June 1972. "f hc Hconomics of Competency- 
Based readier Education/* Herbert Hitc, Research Foundation 
of the State of New York, Albany, N. Y., 1972. 



study materials. Full-time students rather than 
number of classes arc the bases for measuring 
costs. Tfiis is not necessarily a highrr cost fac- 
tor, just different. . ^ 

6. In CBTE, school personnel have a significantly 
larger role in the instructional process and the 
cost of their participatiot\ is relatively higher. 

7. More instructional time is involved in assess- 
ment of student progress, and this process is 
usually more complex than grading in courses. 
Evaluation costs are usually higher. 

8. Decision making regarding all phases of the 
CBTE program involves not only college per- 
sonnel but also school and community mem- 
bers. The large amounts of these peopWs time 
which is needed are a significant cost factor, 
and no fund"^ have been available for this func- 
tion in the past. 

9. Specialist or consultant services to assist state 
agencies in program approval may result in ad- 
ditional costs for CBTE. 

10. Compctency*based programs leading to ad-* 
vanced certification vdll be centered in school 
districts, and the development and administra- 
tion of these new programs will be an addi* 
tional cost factor eventually. 

In a study of costs for the new CBTE certification 
standards in the state of Washington, which was re- 
quested by the stateV legislature, two CBTE programs 
were analyzed. * The study suggests that for the time 
being, the following factors are additional costs of 
CBTE relative to vtandard teacher education programs 
in that state! 

1. Add 50 percent to the tosts of the standard 
teacher education to account for the variable of 
individualized mode of instruction. 

2. An additional SO percent will be required for de- 
veloping each new CBTE program. 

3. An additional 50 percent will be needed to pay 
for released time of school personnel involved in 
consortium arrangements — policy making, pro- 
gram planning, securing program evaluation, etc. 

- One of the pro^jrams analyzed in the cost study was des* 
cribed in the paper previously cited. 
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Yhc fea>innHMulalions to the Washington State Leg- 
islaiuiv arc Un the colleges and universities to receive 
double funding for new CWMi programs and the 
school districts to receive fi.nds for consortium activi- 
ties 

Some of the costs for CUTE are the same if the pnv 
gram includes one student or 1, ()()() students. These 
costs include the development of policy councils, the 
certification priK'css, the development of basic compe- 
tencies and assessment pixKcsses. and a basic set of in- 
dividualized materials. Si>me costs var>' acci^rding to 
the number of students: the number of faculty per stu~ 
ilent, the number of laboratories or public school class- 
rooms, the number ,of sets of materials. Some costs 
represcnf quality judgments, such as the number of op- 
tions to be made available to students, the ratio of fac- 
ulty to students, the degree of refinement of structional 
modules and an^essment procedures. 

Obviously, a good deal of the ditTcrence in costs of 
CB I E and standard programs depends upon decisions 
as to the nature of specific CBTE programs. For exam- 
ple, the definition of competency ciui make a difference 
in program costs. If compctcttCyTs JDfiu^das a set of 
teaching performances, thery the focus w^^^^t^'l^p^!r 
highly refined and tested instructional devices to elicit 
teaching behaviors. If the ei^iphasis is upon products of 
teaching; i.e., pupil achievements, then the emphasis 
will be upon the field setting^. for working with learners 
and evaluating the outcomes o^this activity. 

The (*BTE model at Western; Washington State Col- 
lege is based upi>n product >;riteria. The candidate is 
required to demonstrate competency by bi inging about 
^'appropriate" changes i^i the behavior of elementary or 
secondary pupils. The iihajor components of the model 
are an entry phase, in/which the candidate meets ad- 
mission criteria, a Kilowiedgc phase in which he ac- 
quires a repertoire of information abi>ut the substSice 
of what he will teach, a laboratory in which he ac- 
quires and demonstrates knowledge and skill about 
teaching and learning principles and applies these 
skills, and a practicum in which he demonstrates his 
ability to apply principles of teaching and learning in a 
variety of situa 'Us. The laboratory and practicum 
constitute two quarters of full-time study and practice 
in a teaching center. 

The program depends upi>n close working relation- 
ships in the laboratory and practicum among the stu- 
dents, the professors, and cooperating teachers who go 
through a special training program to prepare as field 
supervisors. The basic csscniials of the program arc 
evaluations of entry into the program, diagnosis of 



knowledge necessary to succeed in the laboratory, a 
test of instructional competency as an entry into the 
practicum and modified IM.C.'s as evidence of mastery 
of the basic mod»*l and hence certification. Study mate- 
rials in the laboratory consist of 52 modules and as- 
sorted readings, films, filmstrips, tapes, observation as- 
signments, and mini-teaching arrangements. The 
students design their own study programs leading to 
the IM.C, which is the basic measure of competency* 
hy negotiating learning contracts with the clinical pro- 
fessors.. Study contracts vary. 

After 4 years of partial success and occasional fail- 
ures, the clinical faculty has arrived at a working 
model. The costs of this working model arc heavier 
than the costs for standard teacher education at West- 
ern. The cost studies which have been completed sug- 
gest that even after initial years of program dev^p- 
inent have been completed, Jt^iU^e^rc ^ faotor ( > 4q^ 
add fo the costs r»iti><r^Vdni\^d programs at Western: 

I. About/fo percent additional is needed to fund 
extnr^upport costs, such as visits by campus- 
baj/Q faculty to the teaching centers, and special 
i^ividualized learning modules, 

^^T^bout 25 percent additional i.^ needed for the 
continuous development of the program compo- 
nents — evaluation instruments, adaptatiqns of 
modules, processes for orienting experienced 
teachers to the model, etc. 

3. About 25 percent is needed for the administra- 
tion of each teaching center, including negotia 
tions with school personnel concerning consor- 
tium arrangements. 

This is a modest proposal for extra funding com- 
paretl to other suggestions for funding CBTE. The 
Western model is still developing, but it is based upon 
modest resources. Compared to the Elementary 
•| eachcr Education models, funded in 1968 by USOE, 
rhis is a Model T compared to a brand new Lincoln 
Continental. The present program docs produce the re- 
quired demonstrations of competency, however, and 
there are considerable affective gains over the standard 
program in the views-of faculty and students. 

Still, even a modest program needs somewhere be- 
tween a 50 and a 100 percent increase in funding for 
teaeher education. Where docs the funding come from? 
in the real world, deans of education do not confront 
the legislature or the boards of regents with demands 
for so many extra millions to fund a new progr^itn. 
Colleges and universities have only a few alternatives. 

I hese are the possible alternatives a college has for 
finding the additional resources for CBI E: 
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1. The college might maintain the existhig level of 
faculty and instructional support funding but 
limit the enrollment of students into the pro- 
gram. 

2. The college might require more time or credit 
hours by students who were candidates for a cer- 
tificate under the CBTE plan, thi s increasing ei- 
ther tuitions or student credit hours and conse- 
quent funding. 

3. The faculty might seek an adjustment in the 
formula for support of programs. Field-based or 

vJiib4>rati>ry programs might be funded at twice 
the level of campus-based programs. 

4. The college faculty might seek outside funding 
— grants or fellowships. 

5. Schools might contribute by sharing the costs 
through support of a complementary inservice 
CBTE program. 

6. Colleges might develop graduate programs, with 
their higher rates of funding, as adjuncts to the 
competency-based undergraduate program. 

In Washington, master's level instruction in state in- 
stitutions is funded at 2.3 times the level of upper-divi- 
sion student instruction. Clearly this level of funding 
would provide what the faculty at Western have de- 
fined as their need for CBTE programs. This adjust- 
ment would require action by the state legislature or a 
major and radical change in programing teacher educa- 
tion within the college. Neither possibility seems likely. 

Outside funding is possible, but not very dependable 
as a base for a continuing extra program cost. The col- 
lege could reduce enrollment and maintain the existing 
level of faculty and support programs, but thisis'politi- 
cally impossible at Western, although this may be pos- 
sible at other institutions, A few school districts will 
share costs. In fact, one school district reportedly has a 
budget of $150,()()0 for advanced certificate programs 
under the CBFE model. These school funds," however, 
are usually restricted to payments to school personnel 
for released time. This leaves the college administrator 
where he always is — with the same old system to try to 
manipulate. Basically the game is to generate sufficient 
credit hours to secure the necessary entitlement for fac- 
ulty and instructional support. 

The CBTE program at Western operates in four 
teaching centers. Each center consists of a team of 
clinical professors, students, and cooperating teachers. 
In a center with four clinical professors, here is a strat- 
egy tor generating the necessary student credit hours 
and getting the necessary work accomplished. 

Iwo clinical professors will have eight to 10 new 
students each quarter. As the stude.Us remain for two 
quarters, this means that the instructional load for each 



of the two professors v.:*' be fiill-time students, at 
\h u ii.Tgraduatc kvw'l. Thoc two professors between 
*hw*m generate three and one-third full-time faculty 
equivalent entitlements. 
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One clinical pro * w takes on half as many stu- 
dents, or a total of nine, which is about two-thirds of a 
full-time equivalent faculty load. He is also responsible 
for the administrative matters ot the center, including 
negotiations with school personnel. 

One clinical professor will direct 25 part-time gradu- 
ate students in a four-credit practicum. The students 
are teachers and volunteer to act as cooperating ad- 
junct faculty in the CBTE program. The credits they 
earn may apply to an advanced certificate. Their prac- 
ticum is concerned with development of the processes 
for working with students in the program, including 
evaluation criteria and techniques. The major responsi- 
bility of this clinical professor is program development. 
Ihe 25 students in the four-credit graduate practicum 
general' nne full-time equivalent faculty load. 

The four clinical prjf:ssors, then, generate five 
t r.E. units, which entitles the center to the needed in- 
>rrnctiona! materials and servi<;e*i which are additional 
t</.rhose required in campus-based programs^ Also, two 
KH the prt>lcsst»rs ca^rn student credit hours in the proc- 
ess of ailministcring and developing the CBTE p^> 
pram. 

I he four clinical professors may modify their own 
assignments. 1'hcy arc jointly rcspoasiblclTor 45 fuU- 
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time undergraduates and 25 part-time graduates, how- 
ever they divide the reNix>nsibilities of the teaching cen- 
ter. These figures are only illustrative. 

A vatiation on this model is a center in which the 
undergraduate intern, in his final quarter of demon- 
strating competency, relieves an experienced teacher, 
who becomes a full-time master's candidate. The intern 
is backed up by a first-quarter laboratory student who 
acts as a teaching assistant part time. Both are closely 
supervised by the clinical professor. In this mixlel, in- 
stead of supervising 18 full-time undergraduates, the 
clinical professor is responsible for four full-time grad- 
uate students, four interns, and four laboratory stu- 
dents, in four classriH>ms. 

These figures and load descriptioT^s represent a few 
of the ways in which the CBTE faculty may generate 
the amount of credit hours needed to support the costs 
of CBTE which are above the costs of standard pro- 
grams. The experience at Western indicates that these 
kinds of loads are feasible, given the nature of the* 
CBTE program which has been developed at that insti- 
tution. 

Summary 

Competency-based teacher education is more expen- 
sive than the standard programs. The variables within 
CBTE programs which afi'ect costs are many, and, ad- 
ditionally, there are many kinds of value judgments 
about programs which will affect the level of funding 
needed for a given CBTE program. Funding for a 
modest approach to CBTE, which is largely field- 
based, is possible within the usual formulas for gener- 
ating support for higher education. The experience of 
clinical professors indicates that the credit hours gener- 
ated by full-time undergraduate students and part-time 
graduate students (ciwperating teachers) are sufficie it 
to fund CBTE. State officials could simplify the fund- 
ing problem, however, by supporting CBTE as a 
"high-cost • college program, equivalent to graduate 
study or to study in the other professional fields such 
as health sciences or law. 

It seems clear from initial experiences with CBTE 
that costs are greater than with regular programs. How 
much greater depends upon a number of judgments 
which arc made by the CBTE program designers. 

I'hc high cost factor which is common to all CBTE 
programs is the individualized mode of instruction, 
which is necessary if teacher candidates are to demon- 
strate competency criteria. Other expenditures, particu- 
larly expenditures for program development and for 



management, can vi^ry wildly from CBTE program to 
CB I E program. 

The cost decision may hinge upon the way the fac- 
ulty define competency. If competency is defined in 
terms of specific teaching behaviors which are assumed 
to be related to changes in pupils, then the modules 
and other training materials will need to be refined to 
the point that it is possible to predict a given level of 
performance on the part of the consumer of those 
modules. It follows from that decision that teaching 
modules and strategics must be highly refined. Costs 
can be very high to achieve this end. 

Competency may be defined, however, as changes in 
the performance of the teachiog candidayte^s xdients. In 
that case the specific modules or strategies will be 
effective inasmuch as they assist an individual candi- 
date to implement growth on the part of the candi- 
date's pupils. The emphasis in program development 
will then be upon the field setting, the education of co- 
^ operStiiigf Scfftoor^^ the evaluation of the prod- 

ucts of teaching. The costs for these components of 
CBTE are more likely to be absorbed within the ongo- 
ing school program for pupils and the inservice educa- 
tion for teachers. " * * 

If the CBTE faculty choose to design all their own 
learning modules, the cost^ will be relatively high — 
they could be astronomical. The cost factor seems 
comparable to the costs of developing programed in- 
struction or instructional systems. There are choices to 
be made concerning the quality levels of the CBTE 
program; e.g., the ratio of faculty to students, the num- 
ber of alternative Instructional strate^es, etc. 

New state certification standards which specify com- 
petency or performance by candidates also require that 
the new teacher educarion programs be managed by a 
consortium of agencies, including not only colleges or 
universities but also school administrators, professional 
ass(Kiations, and the public. Released time for teachers 
to participate fully in training programs and fn policy 
decisions is a new cost for teacher education. 

The realistic level for funding CBTE might be to 
compare teacher education to graduate education or to 
other high-cost programs such as in the laboratory 
sciences.* The rationale for shifting the funding from a 
level comparable to academic college instruction to 
tliat of funding graduate ^or laboratory science instruc- 

• * The higher level of funding per student does not mean 
that the total cost of teacher education necessarily would be 
ughcr: prepara;ion agencies could limit enrollment in their 
::BTE programs. 
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tion is thai CBTE is like other high-cost collegiate in- 
struction. CBrE is a highly individualized pri>gram 
and should be supported in the same way as lUher col- 
legiate programs which are conceded to require a 
highly individualized approach. 

Ultimately, the preparation of teachers under a com- 
petency-based model will need sustained financing in 
the same way as t«^acher education programs are pres- 
ently funded. Specific C31'E programs must earn col- 
lege credits. The credits must meet the qi ality stand- 



ards other college credits must meet. The credits must 
be sullicient to provide for the necessary time of fac- 
ulty and for support programs. The additional funding 
needed to implement the individualized CBTE pro- 
grams must come through the existing system of higher 
education. Therefore, the fate of CBTE seems to rest 
with the decision makers who allocate funds to colleges 
and universities, and specifically CBTE will depend 
upon a more generous formula than is now used to 
fund undergraduate teacher education. 
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Assessment of Teacher Performance: 
What is Involved? What is the Cost?* 

By 

BEATRICE A. WARD 



When the term "performance-based'' is used as a 
descriptor for teach er training and/or teacher certifica- 
tion, certain essentia! conditions apply. What the 
teacher does is as important as what he Knows. During 
training, skill development receives equal or greater at- 
tention than knowledge acquisitioh. At the end of 
training, mastery is used as the evaluative criteria 
rather than a list of courses completed and grades re- 
ceived. Information about the teacher's performance is 
obtained throughout the training period and subse- 
quently in the operable classroom in order to deter- 
mine whether the specified teaching skills are being Ac- 
quired and used. Student performance also is studied 
in order to determine whether teacher use of certain 
skills relates to the level and quality of student learn- 
ing. 

Within such a framework, assessment of teacher per- 
formance is multi-faceted. It is conducted for many dif- 
ferent reasons, focuses upon a wide variety of teacher 
and student behaviors, and involves a diversity of indi- 
viduals and institutions. Inquiry into several of these 
facets of assessment is the purpose of this paper. The 
discussion centers around choices that may be made 
regarding assessment and the costs associated with par- 
ticular options. It builds upon the information obtained 
by the Teacher Education Division at the Far West 
Laboratory, for Educational Research and Development 
during 5 years of developing and testing performance- 
based teacher training materials. 

Why Assess Teacher Performance 

Assessment of teacher performance can be under- 
taken for a variety of purposes. Among them are the 
following: 

• To determine the average type of performance 
that can be expected of experienced or inexperi- 

* Invited- paper. Multi-State Consortium on Performance- 
Based Teacher liducation» New Orleans. Louisiana, February 
27, 1973. 



\enced teachers at a local, regional, state, or na- 
\ional level 

• to identify the types of teaching skills a majority 
of. inservice or preservice teachers perform well 
afid those which they need to improve or acquire 

• To establish a minimum acceptable level of per- 
formance for success as a teacher; to identify 
those teaching skills that relate to successful stu- 
dent performance 

• To determine what an individual's level of per- 
formance is at the time he enters inservice or 
preservice teacher training 

• To establish that an individual has mastered one 
or more teaching skills as a result of training 

Two types of data are associated with these forms of 
assessment. Assessment conducted to determine aver- 
age teacher performance, identify how well teachers 
perform certain specific skills, or establish a minimum 
acceptable level of performance builds upon informa- 
tion regarding teaching in general. Samples of teaching 
and teachers may be used as a data base. The informa- 
tion obtained serves as a guide for policy decisions 
such as the selection of the content (skills and knowl- 
edge) to be included in a preservice and/or inservice 
training program and the specification of . the skill and 
knowledge requirements for certification. General in- 
formation of this type also aids in determining how ex- 
tensive an inservice retraining program should be; i.e., 
the potential number of teachers needing training in a 
given skill and the geographic areas in which particular 
types of training are needed. , . 

On the other hand, the datfi base for planning an in- 
dividual teacher's training jwrogram and/or verifying 
thftt a particular individual has acqjlired certain speci- 
fied leaching skills requires detailed information re- 
garding that person's performance,. Sampling proce- 
dures cannot be used. Rather, a profile of skill usage 
along with a judgment as to the quality of use must 
be available for each individual teacher. The specificity 
of the profile may vary depending upon whether the 
pattern of skill use will be used to outline a training 
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sequence or verify competence prior to certification, 
Pupil learning outcomes may or may not be part of the 
profile depending upon the inservicc or prcserviec sta- 
tus of the teacher. But, regardless of the information 
included, the assessment process must help each indi- 
vidual teacher identify his areas of strength and weak- 
ness and plan a training program that will improve his 
teaching perfonnancc. 

Assessment of the individual teacher or assessment 
of a sample . of the teacher population^ therefore, 
evolves from the intended use of the assessment infor- 
mation. Policy decisions can build from a different data 
bas«: than training decisions. Obviously, given sufficient 
time and financial resources, assessment of individual 
teachers could serve policymaking as well as training 
purpi>ses. However, since time and money are limited, 
study of a representative sample of teachers appears to 
be the most cost/etTectivc means for obtaining the data 
needed to make pi>licy decisions. A variety of tech- 



niques may be used to arrive at the teaching sample* 
For example, an adaptation of matrix sampling (Husek 
and Sirotnik, 1968; Lord and Noviek, 1968) might 
provide a means for obtaining information about how 
a broad sample of teachers* use of a wide range of 
skills at the same time ke.eping costs similaft.to those 
associated with an indepth study of a small sample. 

Based upon these guidelines, the ultimate cost of 
teacher assessment is determined, in part, by the pur« 
poses jpor which the findings will be used* A first step 
toward designing and financing asn assessment effort is 
to outline the^questiohs the data arc to answer* 

yVhfkt Teaching Performance Will Be Assessed? 

Determining wl'^t teaching performance will be as- 
sessed also involved several steps (see- figure 1). The 
first is to identify the potential list of skills to be in- 
cluded in the assessment effort. 



Figure 1 

Steps in Selecting Teaching Performance To Be Assessed 

Stepl 



identify Potential 
List Skills 



Use already 
developed 
lists 



Develop 
own 
list 



Step 2 

Define Range 
of Skill Use 



Define level 
of specificity 



Select subject 
matter areas 



Step 3 

Select Setting in Which 
UsQ Will Be Measured 



Classroom 



^Manipulated 
(microteaeh) 
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liientifyhtfi xkills. The diversity of knowledge and 
skills encompassed within, tho teaching act has been 
documented by numerous experts in the field: i^e^ 
Smith, 1971: Houston, 1972; Joyce & Weil, 1972. Yet. 
to date, research has shown few positive relationships 
between teacliing skill and student outcomes (Rosen- 
shine, 1971). As a result, selection of the skills to be 
assessed; for the most part, consists of combining what 
little evidence exists regarding teacher efTccts with the- 
oretical and practical opinion regarding desired teacher 
performance. 

Several efforts to identify critical teaching skills have 
already been undertaken by individuals and agencies 
such as the developers of the models for elementary 
teacher education ( Elementary Teacher Education 
Mtxlcts, 1969), the designers of competency-based 
teacher training programs (Dodl, 1972), and the de- 
velopers of performance-based teacher training mate- 
rials (the Far West Laboratory) / In each instance, a 
thorough review of the literature related to a particular 
aspect of teaching is undertaken, inclass observations 
are made to verify that certain skills are uspd, and. ex- 
perienced teachers, school administrators, parents, and 
others are asked to judge the relative importance of the 



As suggested by Joyce (1973), persons. responsible 
for teacher assessment may opt to use these already 
developed skill lists or they may choose to develop 
their own. If the latter option is taken, the costs pre- 
sented in table 1 indicate the expenditures in personnel 
and other operating expenses such as travel and pur- 
chase of supplies and materials that may be required. 
The figures are based upon costs incurred by the 
Teacheb Education Division of the Far West Labora- 
tory while identifying critical teaching skills in the 
areas of pupils* language development, mathematics tu- 
toring, independent learning, and higher cognitive 



questioning. On the average, skill identification cost 
$ 1 4.670 for each set of skills. 

The efficacy of using and/or building upon existing 
skill lists is apparent given these costs. Since teaching 
incorporates multiple sets of skills of the type 'dentified 
by the Laboratory, the total cost of skill identification 
could be expected to exceed $100,000 if started from 
the beginning. This would be the cost even if the 
search were to be limited to available research infor- 
mation, and theoretical and experiential opinion. If fur- 
ther research to identify relationships between teacher 
and pupil performance were included, the costs would 
be much hi^er. Thus, if assessiment costs are to be 
kept within reason, the skills to be assessed should be 
taken from the skill lists being compiled as part of the 
broad performance-based teacher education effort. 
Each assessment agency should not attempt to develop 
its own unique skills list. 

Range of Skill Use. The second step in determining 
what teacher performance will be assessed is to define 
the range of skill use to be studied. By this,^I mean the 
leyel of specificity at which the skill will be assessed 
and the variety of , subject matter areas in which the 
skill samples will be taken. ' 

Many areas of teaching can be described in terms of 
general performance levels as well as specific categories 
within these levels. For example, the general teaching 
skill of asking higher cognitive questions can be di- 
vided into the more specific skills of asking analysis, 
synthesis, and evaluation questions (see table 2). 
Within any assessment process, whether it is for policy- 
making dr training purposes, measures of this skill 
probably would focus upon the general performance 
level. Previous research indicates that higher cognitive 
questions generally represent less than half the questions 
asked by teachers (Gallajgher, 1965; Davis & Tinsley, 
1967; Guszak, 1967). In order to set training priorites 



Table 1« Costs of Identifying a Set of Teaching Skills 

Skill Area 



Personnel Costs 
Direct Costs 

TOTAL 



\Pupils' 
Lan^age 

ment 



$ 5,584 
6,938 

512,522 











Math 
Tutoring 

V- - 


Indepen- 
dent 
Learning 


Higher 
Cognitive 
Questioning 


Mean 




$ 4,953 


$ 5,981 


$ 6,647 


11,836 . 


6,156 


7,163 


8,023 


$21 .904 . 


$il , 109 


$13,144 


$14,670 
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Table 2 
Higher Cognitive Questioning''' 

General Skill Level: Use higher cognitive questions 

Specific Skill Jievel: 1 . Use three types of analysis questions. 

— questions requiring students to think of motives or 
causes 

— questions requiring students to make inferences 
' — questions requiring students to find evidence to support 
generalizations, interpretations, or conclusions 

2. Use two types of synthesis questions. 

— questions asking students to make predictions 
— questions asking students to develop solutions to prob- 
lems ' - 

3. Use three types of evaluation questions. 

— questions asking \ students to take a stand on a con- 
troversial issue \ 
— questions asking students to judge truth or validitv 
— questions asking stjudents to judge beauty or wortn 



and/or verify competence in this area, therefore, the 
single measure of percentage of higher cognitive ques- 
tions asked is adequate. If a teacher (preservice or in- 
service) asks more than 60-70 percent higher cognitive 
questions, he is performing much better than the aver- 
age teacher. If he is using considerably less than SO 
percent higher cognitive questions, he probably would 
benefit from training. Given this general piece of as- 
sessment information, many de^sions can be made. It 
is only after a decision to design and instigate training 
has been made that the specific skill levels become im- 
portant. Learning to ask analysis, synthesis, and evalu- 
ation questions can help increase a teacher's overall 
use of higher cognitive questions. Specificity at 
training stage is essential. For assessment purposes, 
general measures of skill use can provide adequate in- 
formation and at the same time keep measurement ef- 
forts and costs within reasonable limits. 

As outlined in figure I , defining the range of skill 
use to be assessed also includes selection of the subject 
matter areas in which teacher performance will be 
studied. Again, this is a decision that affects assessment 
costs. Single examples of teaching are less costly to ob- 
tain than multiple examples. 

For some teaching skills, the decision is obvious. 
Skills related to teaching reading should be assessed in 
the context of a reading lesson. Likewise skills specific 
to the teaching of mathematics need to be measured 
while mathematics is being tau^t. 

* Taken from Mini-course 9/ Higher C ognitive Questioning, 
developed by Far West Laboratory. 



c!(ther skills are more general in nature. Probihg, |or 
exai^iple, can be used in any subject area. It emjploys 
the si^me type of teacher-pupil interaction regardless of 
the cbntent of the lesson. A teacher's use of probing in 
any subject area probably would be representative of 
his us^ qf the, skill in general. 

Stilly other skills differ in use depending upon the 
context in which they are applied; for example, re- 
sponse \ to pupils' errors in reading probably differs 
from response to errors in science. 

The problem within an assessment effort is to deter- 
mine wl^ch subject areas are most likely to offer the 
best opportunity for -teachers to exhibit their use of 
each parucular skill. During ^ the initial phases of asr 
sessment, iinquiry probably will be limitecf to generali2«^ 
able skillslor to specific content area skills. Because of 
data colleolion complexity, information regarding skills 
that apply ^to several content areas but differ in their 
use within dach area will follow later. 

Peirformcince Setting. TTie third step In selecting the 
teaching performance to be assessed is to select the 
setthig in which skill use will be measured. Based upon 
Turner's (1^72) six criteria of teacher performance, 
inclass as contrasted with manipulated examples of be«^ 
havior may lie considered; the most common mani- 
piUated exami^le being a microteach lesson. The salient 
^question to be considered is whether a reasonable 
sample of skilllcan be obtained in a microteach setting. 

Since much 'teaching occurs in small group or one« 
to-one tutorial '^situations, land critics of the classroom 
(such as Silbei^man, 1970) contend that large-group 
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instruction should seldom take place, a smalKgroup 
setting, e.g., a microtcach lessoifi^. may be an appropri- 
ate information base for studying a high proportion of 
teaching skills. The disadvantage of such a perform- 
ance sample is that few student outcomes of any cori^e* 
quence could be expected to occur in a single 10-20 
mtfiute lesson. Thus, if study of teacher effect upon 
student learning is to be included in the- assessment 
process, at least some examples of inclass performance 
should be included. It should be noted, however, that 
given the present state of the art of assessment, re- 
searchers (Smith, 1971; Herbert, 1971) question the 
appropriateness of using student outcomes to measure 
teacher performance. Further, since the restricted envi- 
ronment of a microteach lesson, whether live or on 
videotape, also facilitates concurrent scoring of a 
number of teaching skills, this instructional setting 
warrants serious consideration as an assessment tool. 

Cost of Assessment 

The Teacher Education Division of the Far West 
Laboratory has conducted a number of studies of 
teacher change resulting from minicourse training. In 
these studies, an assessment procedure has been used 
that includes 

— assessment of already identified skills (skills were 
identified during development of training mate- 
rials), 

— assessment of both general and specific skill use 
(e.g., both percent of higher cognitive questions 
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and use of analysis, synthesis, and evaluation 
questions were assessed), 
— assessment within a certain prescribed content 
area, 

— assessment within a microteach setting. 

Using this procedure, microteach lessons were recorded 
on videotape and critiqued at some later date by 
trained observers. Only limited, if any, measures of 
student performance were obtained. 

Sample costs for conducting the assessment are pre- 
sented in table 3. They aie based on the study of five 
teachers because this is the number of teachers who 
can conveniently be scheduled into a microteach facil- 
ity during a 9:00 a.m. to 3:30 p.m. school day. The 
average cost per teacher for five teachers is $180. 
Since critiquer training, research design, and data prejv 
aration costs would not be repeated with a larger sam- 
ple of teachers, the cost to assess 30 teachers would be 
$2, 1 60; Qr $72 per teacher. 

If the skills to be measured and the lesson content 
were carefully selected, such an assessment effort could 
provide a large amount of information about teacher 
performance. It could be used for many of the pur- 
poses outlined* earlier in this paper. 

Who Will Do the Assessing? 

Assessment of teacher performance generally will be 
conducted by three types of individuals. 

1, Teachers. Peer observations (one teacher ob- 
serving another and vice versa) can be used to 
study a wide variety of teaching skills. 
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Table 3. 

Cost of Assessing Teacher Performance 
Using a Microteacli Setting'^ 

Collecting Perfotmance Sample , 

5 l/2-hour yideotapes @ $I5/e»ch $ 75.00 

7.5 hours of research intern to monitor collection of 
video tape examples (@ $9,450/year + benefits) 50.00 



Critiqiung ^Performance 
Critiquer training; 
1 hour training per behavior 

12 behaviors X 2 critiquers X 1 hour . 

24 hours X $3.50/hour ^84.00 
12 hours of research intiern to do training 77.00 

Critiquers to score recorded lessons 
12 behaviors 6 passes to score 
6 passes @ 30 min. each — 3 hours per tape 
3 hours X 5 tapes 15 hours @ $3.50 per hour 
15 hours @ $3.50 per hour X 2 critiquers 105.00 



0 4 

P 



$120.00 



.266.00 

Research design and data interpretation 
} week research design planning 
@ $17»000/year (Note: 
If more teachers^ were involved^ this cost would be 
dispersed' over the^entire population) 434. 00 434. 00 

Qpta preparation • * 

Keypunch ~ ^ day clerk/analyst at $9.,000/year 

benefits , 40.00 

Run and analyze - 1 day 40.00 



8Q.00 

t6tAL cost (Assessment of 5 teachers) $900.00 
* Based on assessment' of 5 teachers. 



Critiquers. Analysis of audio and/or video tape 
recordings of ^caching (e.j,.» a microteach tes- 
son) may be done by spcciaiPy trained critiquers. 
They also'mav be teachers but more often they 
win be gradus(ie or undergraduate students in ed- 
ucation hired to carry out a particular critiquing 
'task. 

Inclass observers. Whenever an assessment pro- 
gram demands high accuracy in recording 
teacher inclass performance, specially trained in- 
dividuals will be needed. Inclass scoring of 
teacher performance requires simultaneous moni- 
toring of multiple variables and instantaneous 
recognition of the skill (s) to be assessed. Con- 



siderable training and pfractice must be com- 
pleted in order to achieve this degree of observer 
skill. 

Information obtained through peer observations will 
be the least accurate form of assessment data. None- 
thclcss» when the assessment is being done to identify 
> inservice teachers* training need^ this type of informa* 
tton is sufficient to cstabli^ a tentative skill profile. 

In our work at the Far West Laboratory, we tested 
a form of peer assessment in the responsive skills area. 
Six observations were made (in this case, four were 
done by teacher peers and two were completed by the 



114 . 

teacher's students). Gross information was obtained re- 
garding such skills as teacher response to pupil ideas, 
teacher use of positive and/or negative r«sptmses to 
student performance, antr teacher use of nonverbal 
reinforcO'ment. Given .the observation findings, the 
teachers wer€ alerted to skill areas in>hich they needed 
to improve. Decisions could -be made regarding 
where to begin the inservice trlfining program for that 
:group of teachers. TRe gross information received from . 
the peer observations allowed the training to proceed • 
with some degree of individual focus. 

Critiquer analyas ©f audio or video tape recordings 
pf teaching samples provides more precise •information 
about the skills used. The data are more reliable than 
peer observations. Many skills can be scored from a 
single teaching sample by rerunning the recording sev- 
eral times. On the other hand, since the teaching situa- 
tion being studied is ananipulated, questions <jnust be 
asked regarding the generalizability of these findings to 
the teacher's regular performance in the classroom. 
The best time to use this form of assessment may be at 
the end. of training when a teacher needs to be sure 
that he has acquired and is using a particular skill or 
set of skills and when nuances in skill use need to be 
pursued. 

Overall, the'most comprehensive observation proce- 
dure is to have highly skilled observers score teacher 
and pupil performance in the classrooin setting.' As 
noted above, this requires the observer to identify spe- 
cific skills at the moment they occur. Teacher-pupil in- 
teraction in an on-going classroom setting cannot be 
.rerun to check the observer's scoring. Further, since 
' the observer can code the use^of only a small nuriiber 
of skills at a time, several observations may be neces- 
sary in order to code the same number, of «kills that 
could be scored in a single 30-minutc video taped les- 
son. Inclass observations appear to be piost useful 
when pupil outcomes are to be assessed as well as 
.teacher performance, when a particular teaching skill is 
expected to occur only in a natural selting, when data 
on teacher performance must be analyzed as soon as 
they are obtained, and when equipment to record 
teaching samples is in limited supply. Extensive inclass 
observations are not essential to five of the six assess- 
ment purposes listed easier in this'paper. 

The cost of assessment depends upon both the type 
of performance to be measured and the assessment 
procedure to.be used. Peer observations are, the least 
expensive; inclass observations the most costly unless • 
large amounts of equipment must be purchased to in- 
stall critiquing of recorded lessons. Table 3 presented 



an example of the costs associated with assessment of 
niicrotcach samples of teaching. Estimates of the co^t 
of trained ob.s6rver scoring of the same number of 
teaching skills in the classroom would be higher than 
the $72 per teacher projected for the microteach sam- 
ple. Because of this increased cost, inclass observations 
probably should be used only when no other "tosess- 
ment method will provide the needed information. 

Developing the Teacher*s Skill 

A basic assumption of teacher assessment is that 
once data are available" regarding a teacher's perform- 
ance, tpainiog will be offered to improve that teacher's 
competence. Using his skill profile to idejp^ify areas of 
strength and weakness, the teacher may waive graining 
in one skill area to emjAasize improving another. To 
be successful, a program that allows these options in- 
corporates training packages that cover diversity of 
skills. Within a performance orientation, each of these 
packages generally will include models of the skills to 
be acquired, opportunities to piactioe the skill^i and 
evaluative feedback regarding the teacher's perform- 
ance during practice. 

As noted by Joyce (1973), development of such 
training materials is costly. T^me and effort, repre* 
sented by dollairs, must be spent to identify the skills to 
be included in the package, develop the initial form of 
the materials, test the materials to be sure teachers ac- 
quire the specified skills, and revise the materials based 
upon testing results. Table 4 jwesents cost da1|i for the 
design of.one such type of training package, the mini- 
courses developed at the Far West Laboratory. The 
data represent average costs for eaqh development task 
based upon 'the desig^ and testing of four minicourses. 

The merit c:' building a training system tlwt uses as 
many already developed materials as possiW^, Is ob- 
>vious given such development costs. 

Tabl& 4. Cost of Developing Training Materials 



Development Task 

Conceptualization' and skill-identification 
Product development 

Product testing (one test with 30-50 teachers) 
Revision after testing 

TOTAL 



Cost 
$ 14,670 
28,962 
46,279 
13,627 

$102,538 



An -additional cost factor that should be considered 
asypart of a complete assessment cycle is the cost of 
providing training. Table 5 lists the estimfted costs for 
a 5-weck training sequence' that includes microteach- 



Tables. Cjwtpf TralnteR* 

t 

^ Siarhup costs 

Mater ialsj>urchase 

I6mmlii(ned models df teaching skills 
teacher handbcH)ks 30 $2. 5() ' * 
video tapes for micrt)tcaching M) 1, 2 hoiir tapes 
Equipment purchase 
VTR equipment, 3 sets 



S I. (XX) 
75 
. 450 



T ramhiK costs 

Coordinator of training. 
, 6 weeks for planning and monitoring 
* ♦Substitutes to releast teachers. 
1 substitute per U) teachers X 5 weeks 
6f traiaing - 30 days (a $35/day 



6,000 
Subtotal S7,525 

2.025 

1,050 

Subtotal 3,075 
TOTAL $10,600 



* Based upon minicotirse training costs. ' ^ 

** Optional; other ways may be used to release inseryice teachers 
tbr microteaching; not required in preservice. 

ing. Start-up costs such as purchase of training mate- 
rials and video equipment are included as well as 'the 
cost of conducting the training itself. Once the training 
materials and equipment are available, this type of 
training- runs approximately $100 per teacher. Given a. 
shorter training period, a larger number of teachers to 
work with the coordinator, or less microteaching, the 
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costs wpi^d^^e.lcss. Ultimately, the cost would depend 
upon whether the training was. part of a larger per- 
formance-based assessment and training system ot a 
»single, one-shot, effort. 

Summary * 

In Ithis paper questions have been raised regarding 

— ;thc purposes of assessment; 
* — the teacher performance to be assessed; and 

— the individuals who will do tlje assessment, p^sirtic- 
utarly the ievel^ of training an^skfll they must have 
based upon the assessment approach b^ing used- 

In each question'^area, sSveral optional, courses erf 
action were presented. Different cost levels were associ- 
ated with each option. Thus, the task for* the individ- 
ual, or agency, responsible for assessing teacher per- 
formance is to determine which options within each 
decision area best meet the^ policy-making and tri^ining 
information needs of that particular program (agency).^ 
Centralno thi^ decision will be the quantity and quHlity ' 
oj[ data required given the .purposes for which the find- 
ings wiir^be (ised. A reasonable balance must be 
achieved between the desire to obtfhn detailed, highly 
reliable data and the total dollars to b^ spent. 
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The Teachers Vie^^ip^nt 

By 

SANDRA FELDMAN 



f have been asked to address my comments specifi- 
cally to the concerns of State Education Agency per- - 
Sonne! who have been and will be making significant 
policy decisions concerning performance education. I 
intend to do justjtliat — atvd I hope youll forgive me. 

Although it doesn^t seem tu be, it should be a source ^ 
of Worry to state educati6n depaifments that teachers 
are suspicious and wary of theiii« that they are not 

^ looked upoa by the profession as allies, that they are 
not looked To . as a resource for the solving of educa- 
tional problems, that they arc not thought of as'provid- 
ing educational leadership. 

Itjhould be food for thought, at least, that often the 

X pohcyihakers in state education departments, in the 

^business of prescribing for the schools, find themselves 
on oppo<!Ue sides of the barricades from teachers in 
legislative and/other controversies on school matters, 

Arapging from certification to teacher education to "al- 
ternative'* school plans or performance contracting, or 
vouchees, ui teacher evaluation, on professional prac- 
tices acts, or questions of school structure and gover- 
* nance— or a host of other issues. * 
V/hy? 

Why don't state education departments^ have a 
healthier respect for classroom teachers, for their accu- 
mulated experience and expertise, for ^their organiza- 
tions as^an expression of teadier concerns? 
. Why aren*t teachers more significantly involved in 
the process of decision-making at state education de- 
I^artment levels. (And by teachers I do not mean the 
state education department personnel v^ho once* wercelO'^ 
the classroom and feel themselves able, therefore, to 
speak for teachers. Nor do I rtean selected individual 
teachers, handpicked b^ school administrations. 1 mean 
teacher leaders selected democratically by the teachers 
themselves, through their organization.) 

I would like to hear what you think the reasons are: 
I think they are manifold and I will tell you what I* 
think first. • >^ 

One reason, in my opinion, is an incomprehensible 
inability of education administration to accept thcf fact 



that times have changed, that teachers are organized, 
that they insist on speaking for themselves and on hav- 
ing a voice in, the decisions which affect not just their 
working lives, but the education of the children they 
teach. 

Insofar aa your invitation to this conference of 
teacher representative^^ den^onstrates some conscious* 
ness of that, I congratulate you — and I intend to take 
full advantage of this opportunity. But I do so with a 
certain skepticism; as a participant in many confer- 
ences such as this one, I seldom see the fruits of such 
exchanges in concrete application back home. 

Another reason for a lack of real cooperation be- 
tween* teacheVs and their state education departments is 
a result, I thinks of some sti^te education department 
officials seeing themselves as in the vanguard of educa*- 
ttonai change and seeing teachers as defenders of the 
statils quo. .To teachers, this characterization is ludi- 
crous. They do not see their deiHands for drastically 
, reduced class size, for massively ^inV^ased social, psy- 

* chological, and health services for\:hildren, for the 
maintenance of high professional standards as ''old 
hat.*' They do not §ee performance c^jitracting, **ac- 
countability,'' legislation like the Stull i^ct, differen- 
tiated staffing, and merit pay schemes as revolutionary 
improvements, but as* backward steps and attacks on 
the integrity of the profession. They are not- inlpressed 
by paper-weighty master plans or glossy brochures ad- 

, vertlsing "b^d new steps'' which usually do nothing 
whatsoever to help them in the classroom. 

There is yet another reason, in (ny opinion, for the 
"credibilit^^p** between teachers and state education 
departments; and this one I will give a tittle more time 

* to, for it win lead me to the subject at hand. That is, I 
fear that many ftate educ|tion department officials in 
policymaking roles (l)-are under great pressure to re- 
spond to the now famous "crisis of confidence" in the 
schools, without the knowledge and resources to do 
sdmething real to end failure, and (2) share in the 
growing public attiti|<ie that teachers are responsible 
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for schiwl failure and either don't know how to do 
something about it — or don't want to. 

Si>. if we are to succeed in achieving the very wor- 
thy goal of this consortium— ivhich is sttpposcd to be 
"devoted to improving the qsatity of State Education 
personnel and their decision-making abilities.'* we have 
to first establish that one important, essential way of 
improving that policy decision making is to listen to 
the teacher organizations. Then we have to talk about 
the crisis in confidence that spurred the performance- 
based movement — why we have it and what we ought 
to do about it. 

% 




Sandra Feldihiaii 

Crisis in Confidence ^ 

The schools do not appear to be meeting the de- 
mands of a changed economy, of a job market which 
requires moie and more skills and increasingly higher 
levels of training. Whether or not the schools in the 
past were the vehicle for upward mobility, for provid- 
ing the path cut of poverty, most people believe that 
they were — and expect them to continue that function. 
This problem of higher expectations is compounded by 
the^ social turbulence of the sixties and by the racial 
conflict which so often was most visible around school 
issues. 

Second, despite the recent "revelation" by Christo- 
pher Jencks that schooling cannot end economic ine- 
quality among the adult population — and despite the 



fact that this was undoubtedly always true, that eco- 
nomic equality would have to be legislated as a matter 
of swial pi>licy — public schooling has undeniably pro- 
duced a literate citizenry. Where in the past — and es- 
pecially through the thirties, forties, and fifties — school 
teachers were among the most educated people in our 
society, even when they only had a high school or 
training school degree, today, the country is full of ed- 
ucated people. Thousands^ upon thousands of parents 
and citizens in white collar government employment, in 
advertising, in media jobs, accounting, computers, in a 
wide variety of nonteaching academic pursuits, feel 
that if they were not working at their present occupa- 
tion, they could teach. Today, although teachers are 
still among the most educated,- they do » not have the 
status that comes.with having knowledge that others do 
not h^ve. While a holder of a Ph.D. in economics feels 
that a doctor's or a lawyer's skills are very special, 
even somewhat mysterious and intimidating, he does 
not feel that way about a teacher— even if the teacher 
also holds a Ph.D. 

As a profession, education must, like other profes- 
sions, assemble its experience and skill into a concrete, 
coherent body of knowledge as tangible as what exists 
in law or medicine. 

That is why we must have performance-based 
teacher education. Not only will it make teacher educa- 
tion more relevant to classroom needs and thus im- 
prove education, but it will, if developed properly, pro^ 
vide us with that concrete knowledge without which we 
cannot much longer defend the public school system. 

And that is why I believe, unshakeably, that 
developing a body of knowledge about the teaching- 
learning process ought to be the initi-' thrust and 
prime, concern at present of the performance-based 
teacher education movement. We should be setting 
about doing what Fred MacDonald is proposing be 
done by a National Commission on Research and De- 
velopment. We should be making a coordinated, long- 
term commitment to validating teacher competencies 
■ — not what is being done, which is a short-term com- 
mitment to listing them. We should be working at 
proving what teacher behavior, what teaching strate- 
gies, effect what learning and how. 

We should not, as the New York State Education 
Deparunent is doing, be telling teacher education insti- 
, tutions to "Poof! Change over to a 'performance-based 
teacher education program for. certification approval 
purposes." At a meeting several months ago, to which 
representatives o*f teacher education institutions and the 
state teacher organization were invited, there were as 
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many different notions of what • performance-based 
teacher education is as there were deans of education 
at the meeting. And the leadership in the state educa- 
tion department is not providing a model or a research 
design or any roal guidance—unless you call require- 
ments listed on a form for program approval, guidance. 
We will have as many ditTerent, and as many irrelevant 
and inadequate teacher education programs as we 4^) 
right now — but New York State will ••brag" about \x» 
performance-based teacher education program at con- 
ferences like these throughout the country. 

Just recently^ at a meeting of sophisticated chairmen 
of educatiiuv departments of one of our lafgest teacher 
education institutions, I saw education department >rep- 
resentatives of. a college struggle with a paper descriipi-' 
tion of what they hoped would be considered a per- 
formance-based program; and they had listed no^ 
teacher competencies, but behavioral objectives — which 
is fine to do, but from which^ as' Bunny Smith has em- 
phasized over and over, you learn nothing about the 
teaching process. ' 

I have grave doubts as to whether the states can au- 
tonomously develop meaningful performance-based 
programs — but I have no doubt that most teacher edu- 
cation institutions, without resources for research and 
without access to a large variety of classrooms, cannot 
do it. 

If the performance-based teacher education move- 
ment is going to be meaningful, it cunnot be handled 
as a public relations gesture by state education depart- 
ments and tossed back to the colleges |which have been 
floundering alt ak>ng. vNor should the jeffort to develop 
a knowledge base of teacher compet(|ncies be tied to 
certification a^t this time, and certainly not to continu- 
ing certification schemes^ because teachers will not 
stand for the destruction of tenure ; and job rights 
through renewable certification — especially when a re- 
view of performance is based on unvalidated list^ of 
competencies. \ 

If performance-based teacher education is going to 
bo meaningful, it cannot be — as every education "inno- 
vation''' usually is— doomed, to success. In any other 
profession — take medicine — strategies jare tried and 
tested and discarded if they don't work and the knowl- 
edge gained from systematic research into those strate- 
gies, whether they cure the disease or not, provides a 
continuous' buildup of intelligence in the fieldv: In edu- 
cation, we scurry to hide — or ignore-— failure. That 
the research weVe done so far is inconclusive, because 
sophisticated instruments to successfully interpret and 
measure teacher behavior are still in the eariy stages of 



development— and I know you vc heard MacDonald, 
Rosenshine, and Schalock — should not be a source of 
embarrassment to us. It should provide us with the re- 
solve to push that important work forward. 

Instead, each state goes its own way, without a pool- 
ing and sharing— not just of knowledge, but of politi- 
c:il power to ge a commitment for funds to do the 
usearch we all know needs to be done. In New York 
State we have a very definite timetable. By — is it 
1984? — we will have performance-based teacher edu- 
cation, so-called, and performance-based teacher certi- 
fication throughout the State. But will we know any 
more about education or about teaching? I think not 
— unless the National Commission for Research and 
Development is ready to share their work with us-*- 
and they haven't got neariy enough money to do the 
work that has to be done. ^ that is the direction we 
should be taKing. ' [ 

What w^'should be doing — and actually the multi- 
state consortium might well be an excellent vehicle for 
this — is coijJ'dinating a single effort, a national effort to 
develop a knowledge base. We should be collating the 
limited knowlcfdge we now have, and we should be 
building a small number of teacher models based on 
available research and the opinion of experienced 
classroom educators. We should be developing in a 
broad cross-section of schools throughout the Nation a 
systematic assessment and data-gathering machinery 
which would enable us to compare and study teaching 
behavior and its eflfects on learning response in a pre- 
scribed variety of school environments. ,We should 
teach prospective teachers in the models created — not 
hundreds of different ones, but a few — so that we can 
control for effects, and we should have a research de- 
sign built in so we can validate the competencies we 
have agreed on by studying those new teachers, on the 
job, where their education should continue during an 
internship-probationary period. 

While I believe that the main thrust of perform- 
ance-based teacher education should, in the beginning, 
be preservice, there must be involvement of experi- 
enced classroom teachers because they have a great 
deal of expertise to offer and because they will learn a 
great deal in the process of participating. 

This past summer, The City University of New York 
came to UFT to. seek cooperation in the dev)»lopment 
of competencies for teacher-training models. In a short 
time, we recruited over 300 interested teachers (who 
were paid a fee for their participa^gn* as they should 
be). They provided a valuable contribution to the* 
work, and they learned 9 great deal: 
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•i was forced to think out iiva concrete role my role 
and skills as a teacher." 

"I was fowed to analyze myself and thhik o'* what 
really makes a good teacher." 

"It is high time an emphasis was put on specific 
competencies rather than abstract thet>ries. " 

"I came away more optimistic about the future »)f 
education." 

• (And school critics are always seeking a way to bet- 
ter teacher "attitudes"! ) 

State education departments should be joining with 
the teacher org Jnizaiions to demand that National In- 
stitute of Etiucation and its funds be put at our dis- 
posal; that it be used as a resource for the profession, 
not in the form of hundrc^is of diverse grants awarded 
in a scattered way throughout the country, but in a 



njassive. concentrated effort to develop the profession 
in the way I "have just briefly described. 

.'\s state education department personnel, youshould' 
not be plunging forward blindly, as I think ytS^ are, 
because of the gfeat public pressure to "do stmicthing." 
You shoultl insist on your right and your nee<J to base 
your decision making on substantive, proven, profes- 
sional knowledge. Whether you do or don't, the' 
leacher organizations will insist that you do; jjjd^tf; 
there is a struggle between those demanding^^iclue, 
public relations solutions and the teachers wKoare de- 
manding substance, you will be squeezed in the middle 

unless you have already chosen sides — and the 

P'omise of performance-based teacher education may 
die in the process. 
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Some Substantive Issues And Political Considerations 
In Performance-Based Teacher Education 

By 

BERNARD H. MCKENNA* 



mat Have We Heard? 

We*ve heard here over the past 4 days that 

I. the state of the art is little advanced — in fact 

someone termed it **primitive**; 
2. there is little data on what makes for compe* 
tence — as some put it — ^'Wc know too little 
about what's worth teaching*'; 

3. the most difficult j[oU will be defining competen* 
cies; 

4. adequacy of assessment apparatus is the most 
critical issue and will be most difficult to de- 
velop; ^ 

5. the definitions of behaviors are» at this' point, 
almost totally hypothetical; 

6. we lack* summative evaluation tools; i.e., there 
are rating forms for making global judgments, 
and there are research tools containing specific 
skill fonriulation — ^but nothing in betweeii; 

7. there is a complex interaction between the set- 
ting and performance, an interaction that af- 
fects performance in a major way, one that vf& 
haven^t much taken into account, let alone ana- ^ 
lyzed; 

8. the affective domain is being neglected in per- 
formance-based teacher education; 

9. the tools for measurement in the social sci- 
ences, including teacher education, are in an 
iiifant stage; i.e., even with the sophistication 
developed in measurement on cognitive skills 
such as reading, weVe still not abTe to diagnose 
causes of all reading difficulties, let alone pre- 
scribe behaviors that will correct them — so« it 
has been asked, how will we soon be able to 
measure attitudes, values* and the like? 

10. there is a monum^tal problem in matching 
teaching styles to learning styles; 

^ Dr. McRenna is professi<»mt associate. Instruction and 
Prcrfessional Development, National Education Association. 



II. finally we've heard that measuring specific per** 
formances at various criterion levels will be 
costly. 

There are a lot of other lesser problems we've heard 
about, and likely some additional major ones I've 
omitted. . 

What Does What We Ve Heard Say To Us? 

One thing it mi^t say is ''for^t it." My starting off 
with such a laundry list could cause you to believe 
that's where I am — "forget it," or that Fm launching 
into a defensive diatribe. ' " 

Actually, I believe that performance-based teacher 
education is a promising and viable concept and should 
be pursued. 

My listing these concerns is for the purpose ot put-* 
ting a framework around what Vm about to les'pond^ 
to, and that is the foUowiug questidh. 

What Are the Main Issues in Performance-Based 
Teaciier Education That Rescarche^^ 
Developers, Teachers, Professors, and 
Others Should Be Emphasizing? 

In my judgment, a first line of acdvity should be 
around tools for evaluation. Performance-based teacher 
education will nse or fall on our ability to evaluate* 
That is, a first«order question is 'How shall/ we deter- 
' mme that teacher candidate A is now reacfy to practice 
the profession and should be licensed to do so and that 
candidate X should .'*^ back to the drawing board"?* 

Another way of saying it is that until we can meas- 
ure performance in such a way that we are able tu 
confidently identify minimal levels of performance to 
practice the professior^ we will not have performance- 
based teacher educatifib. 

Some have said at this conference that prior to i^n- 
ttfying minimal levels of performance there are the 
more difiicult jobs of 
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(a) establishing goals and objectives for education^'' 

(b) identifying performances * (competencies*) re- 
quired for achieving the goals and objectives, j 

1 don't believe those to be the most difficult tasks; 
and even if they were, we're further along in accom- 
plishing them than we are on the evaluation matter. 

Forcxamp'**. there are available 

1. some well-devi;loped needs^assessmcnt systems; 

2. several quite complete taxonomies; 

3. the works of Mager, Aikin, Popham, and others^^ 
on getting goals and objectives intd more nearly 
manageable performance-based terms; 

4. . '*The Florida Catalog of Corilpetenccb" and 
"Sigilsr of Good Teaching"* (the latter is less 
known but highly developed) ;' 

5. tW Seven Cardinal Rrinciples or Ten Purposes 
of Secondary Education; . 

6. the techniques of most recent vantage reported at 
this conference by Adele Thomas of The City 
University of New York in which they were able 
to gain high consensus on which skills should be 

- most valued; 
7^ the Northwest Regional Laboratory materials 
^ and those of the Teacher Corps. 

rm not that concerned about our ability to arrive at 
goals and objectives. 

Some in this conference have argued that "instruc- 
tional systems/* as Bruce Joyce termed them, will be 
most difficult to dcvelqp. Vd argue that wp have made 
quite satisfactory progress on these also. 

Whether one goes with those toward the affective 
end of the continuum like "Man a Course of Study** 
and the Taba Social Studies Curriculum or the more 
behavior-modification orientcjd types reflected in some 
of the packaged curriculums, there is a variety of 
quite well-developed instructional systems to choose 
from which include both content and teaching strate- 
gics. 

So that brings us back to evaluation devices — and I 
repeat — performance-based* teacher education will rise 
or fall on our ability to evaluate performance in rela- 
tion to agreed-upon standards. 

Past research on evaluation leaves much to de- 
sired; and I*m convinced that too little emphasis is 
being given to research and development in this key 
area in present efforts with performance-based teacher 
education. 

There are places to start, and I'm surpris^ that 
"Inore of the programs haven*t attended to those prom- ^ 
ising possibilities. For example, the performances iden- 



tified by, Rosenshine and Furst, such as clarity, varia- 
bility, enthusiasm, have shown some promising,, 
irelalionships to outcomes with students and are one 
place to begin. 

I hcre arc also others that should be considered 
whether or not they show direct relationship to learn- 
ing outcomes. The point here is that there are some 
processes which may not lead directly to agreed-upon 
goals (or at least cannot be demonstraltod to dc^so) 
which should be prompted and which are worthy of 
evaluation for their, own sake. One brief illustration of 
this. A recent study of several thousand students repre- 
sct)tativc of the American high school population con- 
cluded that great numbers of students may be develop- 
ing little affinity to the democratic process simply 
because they have little opportunity to experience it 
during their school career. If this is so, is it not impor- 
tant that the process of education in the schools be- 
come, in as much as possible, a microcbsm of the best 
of democracy as" it is practiced in the greater society? 
And isn't this hnportant whether or not such processes 
can be definitively shown to contribute to specific 
learning outcomes? Evaluation systems for perform- 
ance-based teacher education should incorporate such 
, criteria. " 

I believe, as Bruce Joyce predicted eariier in this 
conference, that it will take 20 or more years to de- 
velop reliable and valid evaluation tools. It will proba- 
bly cost more than his suggestecl $100 million. When 
asked if it were possible to develop an accountability 
model for New York City, Henry Dyer of Educational 
Testing Service, responded that it could be done but 
would be more difficult than getting to the moon— -and 
it is well known that NASA had for several years for 
that project, almost unlimited resources and unlimited 
power. 

Three years later Fred JVfcDonald and his people 
have made some progress ott the New York City ac- 
- countability mandate. IVe just read the description of 
their model It appears promising on first .look. But it 
almost totally skirtr the problem of relating perform- 
ance to student achii^vement. What might be the cost 
and the time line if thjs issue were responded to? 

Nat Gage told us on Sunday that Lindquist gave 
warning 30 years or mdre ago that there are some 18 
or 20 variables other than teacher performance which 
contribute to student achievement. Gene Glass put it 
well recently when he concluded froip his research that 
**aside*from the irrelevance of much of the-^content of 
standardized achievement tests, their use .in evaluating 
teachers is unjust.^ 
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If you don't think that kind of thing is being pro- 
moted, you have only to read the Stull A^t in Califor- 
nia. It's like direeting the medieal profession to find a 
eaneer eure in a Vear or mandating the Nation's econ- 
omistK to eorrcct the balanee-of-payments eonundrum 
in a few months. Or look^it the Fleischmann .Conimis- 
sion Rcpi>rt in New York Statv\ or some of the Manr 
agement by Objeetives programs that are being laid on 
school staffs; aeross the country. 




^ Bernard H. McKenna 

Two other quick things on the evaluation issue. The 
first is that where work is being done> on evaluation re- 
hited 4o performalice-based teacher education, there 
. appears to be overemphasis on evaluation of didactic 
teaching styles. One hears the terms "teaching lessens" 
and "collecting student papers*' and other lecture-type 
activities referred to ail too frequently. I thought Betty 
Ward was very bold when she raised the question yes- 
. terday about the desirability of assessing 'skills of total 
* groups since the teaching process is becoming 'more of 
a team^ arrangement, more interpersonal, more a reflec- 
tion of the open-elassroom concept, more guided inde- 
pendent study. I say she was bold, since the mini- 
courses she has been so involved in developing to a 
con.sidcrable degree assume a didac|ic teaching style. 

The second point is that there ^ould be more con- 
sideration of evaluating staff as a team. I think it was 
Betty Ward also who suggested "doo*t test every skill 
in every teacher." In this regard, we might look on the 
staff as an orchestra — evaluating for complementary 
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skills — not sameness. (An orchestra of all piccolo 
players would be piettv dull and would certainly re- 
ceive a strange review from the music critics.) There 
has been ei>nHlderable research done on this team con- 
cept of evaluation over a period of 30 years in the In- 
stitute ot Administrative Research, Columbia Univer- 
sity. ••Inilieators of Quality'' is promising tool for this 
purpose. 

How Does Perf ofinance-Bas^d Teacher 
Education Relate to Inservice? 

If inservice- cjJucation had ever amounted to 
anything, we'd have had respectable performance-based 
teacher education long ago. 

We've had inservice education a long lime. We've 
also had performance evaluation inservice a long time. 
In fact, one might expect most inservice evaluation to 
be performance-based and for the purpose of determin- 
ing corrective measures (inservice needs). 

Instead this is the way it has been: inservice evalua- 
tion of school staffs has been poorly' conceived, little 
developed, only haphazardly implenjented and the re- 
sults often inappropriately used. 

In our work with 1- million plus teachers across the 
country, we note some quite commonly held attitudes * 

on thein part about inservice education: 

* 

^ 1. They find it threatening. 
2. They assume its main purpose will, be to deter- 
mine change of status-^retention, tenure, promo- 
tiou. ^ 
^ They feel untnfortned about the criteria by 

which they are evaluated. 
4. They assume evaluation ,will be accomplished 
through t:»rief observations (too brief) and based 
on some kind of checklist: 
• 5. They expect it will be done to them by some- 
body else, someone in a more elevated position. 
6. Th«y say. the feedback they get from it is inade- 
quate and often absent. 
^ 7. They believe there will be few concrete recom- 
mendations for improvement as a result of their 
being evaluated. ^ 
8. They expect that following evaluation there will 
^ be iittljs or no help to improve their performance 
through inservice programs geared specifically to 
correct the lacks identified through evaluation. 

That's the way it is according to our members. So if 
iX'rCormance-based teacher education is to be carried 
ovc*r to inservice education, it will need to respond to 
all the deficiencies of present inservice evaluation and 
inservice education. And that's a big order. 

If performance-based teacher education works, with 
its proposed field orientation, hospital-schools concept, 
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internships, and career ladders, then the transition 
from prcseryice to inservice both in terms of the proc- 
ess and the evaluation of it should become so gradual, 
»v, smooth, that one can't tell where one leaves off and 
the other starts. It would then approach what the law 
profession is getting underway in the new Antioch Law 
School in the city where I live, and what Western Re- 
serve University got underway in medical education 
more than IS years ago. 

That's short shrift for the inservice aspects, but there 
is some other ground to cover. 

What Is Recommendable? 

A. Involve on a parity-basis — not just token — all 

groups within the profession who will be affected 
y the program. You might begin with some of the 
states in this consortium. 

B. Assure that goals and objectives for PBTE are 
based on goals and objectives for schooling. 

C. Assure that goals and objectives for schooling are 
cooperatively developed by parents, educators, and 
students. 

D. When objectives are turned into performance cri- 
teria, provide for macro as well as micro perform- 
ances. 

E. Assure that program and conditions are evaluated 
concurrently with performance and that they are 
related to it. It's futile to evaluate a teacher's per- 
formance without also evaluating all these condi- 
tions that make competent performance possible. 

F. Involve all those who will be affected by.it, in de- 
veloping the evaluation plan— prospective teachers, 
teacher supervisors, their students, and some par- 
ents.' 

G. Use multiple indexes for evaluation. 

H. Involve a variety of personnel in conducting evalua- 
tion — ^self, peers, supervisors, students. 

I. Thoroughly train those who will evaluate. This may 
be the most important. 

J. Develop concurrently plans for correcting both 
those program and condition elements and perform- 
ances which are shown lo be deficient as. a result 
of evaluation. This nia;^ be the most costly, partic- 
ulariy on the inservice side. 

K. Assure that there will be ample opportunity to im- 
prove (includine direct assistance, time, and mate- 
rials) for those judged to require improvement as a 
result of evaluation. • 

L. Provide for full-blown, written, agreed-upon ar- 
rangements for both substantive and procedural due 
process when >evaluation systems are to be used for 
decision making on other than determining .^erv- 
ice; that is, the r<itention, tenure, promotion. *i • 

M. Secure research fui^s commensurate with the large- 
^ness of the task. \ 

N. Develop time lines that allow for realistic accom- 
plishment of what's accomplishable, taking into ac- 
count the state of the^ art and the complex inter- 



relationships of the many variables as well as the 
feebleness of tools in social science measurement. 

O. Researchers get in there and testify before legisla- 
tures on what's possible and what's not. 

P. Stay close to the schools, (i couldn't disagree more 
witn Fred McDonald when he said, "You can't 
learn anything in the messy classroom situation." 
That may be why ETS was so long in recognizing 
the injustices of the National Teachers Examina- 
tion.) . , 

Q. And finally, provide sufficient periods for test and 
tryout, evaluation, and recycling before dissemina- 
tion — 'before making extravagant promises about all 
the ills the new program is sure to correct. Let us 
not allow this innovation to suffer the same set- 
backs as: 

L the teacher aide concept which got underway in 
Bay City, Michigan, 20 years ago, but was prob- 
ably delayed 10 or more years by poor project 
design and overstated promotion; 

2. or educsitional television which suffered some of 
the same problems partially as a result of the 
Hagerstown experiment in the midfifties; 

3. or differentiated staffing which appears to have 
suffered in a similar way in the last 2 or 3 years. 

What Afe The^itics Of The Matter? 

My final charge was to deal with the politics of per- 
fomnance-based teacher education. 

As you have doubtless noted by now, I assumed I 
had several charges. 

Ted Andrews told me by phone that the symposia 
were to be on assessment in CBTE. Later in a letter he 
suggested I deal with its relation to inservice education. 
And when the program came out, the term ••politics" 
appeared in the title of my presentation. ^ 

According to someone's definition, politics is "the 
art of the possible." If that's so, I've already covered 
what I think is possible and what conditions I believe 
are necessary to accomplish the possible. Just an addi- 
tional thought on political con^derations. 

I am weary of power struggles. Unlike Pat Ooralski, 
I don't accept them as givens. I hope soon well get be- 
yond them so we can concentrate our energies on get- , 
ting the priorities of the American people in better 
balance in order to obtain thv conditions and resources 
that are so badly needed for schooling, all the way 
from prenursery school to graduate programs. 

This can be accomplished, at least partially, if we 
work toward parity in decision making and then c(m- 
centrate on issues rather than on which individuals or 
groups will have the most power. 

Redistribution of power so that it will be more equi- 
table among all groups concerned will in the end give 
us all more power. 
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